Commit 9b827fb1 authored by Dmitry Lukhtionov's avatar Dmitry Lukhtionov
Browse files

Merge branch 'patch-5' into 'master'

Update FreeBSD.md, Jails part finished

See merge request noc/tower!42
parents bc7670de 104f3cef
## Preparation ## Preparation
### FreeBSD ### Install tower prerequisites on FreeBSD
```shell ```shell
root@tower:~ # pkg install -y ca_root_nss python27 libffi py27-setuptools py27-pip py27-virtualenv py27-sqlite3 git root@tower:~ # pkg install -y ca_root_nss python27 libffi py27-setuptools py27-pip py27-virtualenv py27-sqlite3 git
root@tower:~ # pw groupadd -n tower root@tower:~ # pw groupadd -n tower
root@tower:~ # pw useradd -g tower -s /bin/csh -d /home/tower -n tower -m root@tower:~ # pw useradd -g tower -s /bin/csh -d /home/tower -n tower -m
``` ```
## Installation ## Tower installation
Tower is installed into /opt/tower directory by default, though you Tower must be installed to `/usr/local/tower` directory.
can use arbitrary directory (i.e. /usr/local/tower) as well.
Replace /opt/tower/ to directory of your choice
- Create Tower directory - Create Tower directory
```shell ```shell
root@tower:~ # mkdir -p /opt/tower root@tower:~ # mkdir -p /usr/local/tower
root@tower:~ # cd /opt/tower root@tower:~ # cd /usr/local/tower
``` ```
- Create virtualenv - Create virtualenv
...@@ -23,18 +21,19 @@ root@tower:~ # cd /opt/tower ...@@ -23,18 +21,19 @@ root@tower:~ # cd /opt/tower
If you're in csh, rehash first If you're in csh, rehash first
```shell ```shell
/opt/tower# rehash /usr/local/tower# rehash
``` ```
```shell ```shell
root@tower:/opt/tower # virtualenv-2.7 . root@tower:/usr/local/tower # virtualenv-2.7 .
``` ```
- Install Tower - Install Tower
```shell ```shell
root@tower:/opt/tower # ./bin/pip install --upgrade pip root@tower:/usr/local/tower # ./bin/pip install --upgrade pip
root@tower:/opt/tower # ./bin/pip install https://cdn.getnoc.com/tower/noc-tower-latest.zip root@tower:/usr/local/tower # ./bin/pip install https://cdn.getnoc.com/tower/noc-tower-latest.zip
root@tower:/opt/tower # chown -R tower var/ root@tower:/usr/local/tower # ./bin/pip install ansible==2.7.11
root@tower:/usr/local/tower # chown -R tower var/
``` ```
- Generate Tower ssh keys - Generate Tower ssh keys
...@@ -44,22 +43,25 @@ root@tower:~ # su - tower -c "ssh-keygen -t rsa -b 4096" ...@@ -44,22 +43,25 @@ root@tower:~ # su - tower -c "ssh-keygen -t rsa -b 4096"
- Run Tower - Run Tower
```shell ```shell
root@tower:~ # su - tower -c "cd /opt/tower/ && ./bin/tower-web" root@tower:~ # su - tower -c "cd /usr/local/tower/ && ./bin/tower-web"
``` ```
If you want to restrict address which tower listen to, add ```--listen=YOURIP:YOURPORT``` to ```./bin/tower-web``` command If you want to restrict address that tower listen to, run `./bin/tower-web --listen=YOURIP:YOURPORT`
## Prepare nodes ## Prepare nodes
On each FreeBSD node do the following:
If you had installed PostgreSQL and MongoDB previously, you have to deinstall them and clean their db paths (`/var/db/mongodb` and `/usr/local/pgsql`). On each FreeBSD node do the following:
* Enable SSH: * Enable SSH:
```shell ```shell
root@noc:~ # sysrc sshd_enable="YES" root@noc:~ # sysrc sshd_enable="YES"
root@noc:~ # service sshd start root@noc:~ # service sshd start
``` ```
* Add ```/var/run/syslog``` socket for ```consul``` if node will run it: * Add `/var/run/syslog` socket for `consul` if node will run it:
```shell ```shell
root@noc:~ # sysrc syslogd_flags="-s -p /var/run/log -p /var/run/syslog" root@noc:~ # sysrc syslogd_flags="-s -p /var/run/log -p /var/run/syslog"
root@noc:~ # service syslogd restart
``` ```
* If node will run postgresql, you'll need to do the trick: add postgresql server as a package first, then build databases/py-psycopg2 from ports with python 2.7: * If node will run postgresql, you'll need to do the trick: add postgresql server as a package first, then build databases/py-psycopg2 from ports with python 2.7:
```shell ```shell
...@@ -78,44 +80,51 @@ root@noc:~ # pw useradd -g ansible -s /bin/csh -d /home/ansible -n ansible -m ...@@ -78,44 +80,51 @@ root@noc:~ # pw useradd -g ansible -s /bin/csh -d /home/ansible -n ansible -m
root@noc:~ # echo "ansible ALL=(ALL) NOPASSWD: ALL" > /usr/local/etc/sudoers.d/ansible root@noc:~ # echo "ansible ALL=(ALL) NOPASSWD: ALL" > /usr/local/etc/sudoers.d/ansible
root@noc:~ # passwd ansible root@noc:~ # passwd ansible
``` ```
* Ansible will use ```virtualenv``` but here in FreeBSD we have ```virtualenv-2.7```, so to not make things comlicated, just add a symlink:
```shell
root@noc:~ # ln -s /usr/local/bin/virtualenv-2.7 /usr/local/bin/virtualenv
```
* Back to tower machine, copy ssh key from tower user to each node: * Back to tower machine, copy ssh key from tower user to each node:
```shell ```shell
root@tower:~ # su - tower -c "ssh-copy-id -i /home/tower/.ssh/id_rsa.pub ansible@192.168.1.88" root@tower:~ # su - tower -c "ssh-copy-id -i /home/tower/.ssh/id_rsa.pub ansible@10.1.1.201"
``` ```
* Check if tower able to connect to node by ssh with keys: * Check if tower able to connect to node by ssh with keys:
```shell ```shell
root@tower:~ # su - tower -c "ssh ansible@10.1.1.201" root@tower:~ # su - tower -c "ssh ansible@10.1.1.201"
``` ```
## Deploying # Jails
Here's what you need to do to run NOC in jail.
* Jail must be configured using VNET network interface, so that you will have a lo0 interface with 127.0.0.1 address on it inside a jail. IP 127.0.0.1 is sometimes hardcoded all over NOC's components, so you will have hard time deploying NOC to jail without 127.0.0.1 address.
* Do all mentioned in [Prepare Nodes](#prepare-nodes).
* Make sure `/var/run` and `/tmp` are mode 777 (just in case).
* Make sure `/etc/jail.conf` have `"allow.sysvipc"` for PostgreSQL and `"allow.mlock"` for MongoDB.
* During deploy there will be SSE4.2 check, which is done by greping `/var/run/dmesg.boot`, and this file will be empty EVERY TIME YOU START JAIL. So you have to copy host's `/var/run/dmesg.boot` to jail's `/var/run` and do deploy without restarting jail (or do this every time you restart jail). You will need this for the time of deployment only. You may add to `/etc/jail.conf` (assuming jour jail root is in `/usr/j/noc/` and your thin jail is mounted to `/s` path):
```shell
exec.poststart = "cp /var/run/dmesg.boot /usr/j/noc/s/var/run/";
```
* If you have thinjails then probably you have read-only root in it, so you have to change `/opt/noc` path to more BSD'ish `/usr/local/noc` path in tower deployment config. WARNING: `NOC` MUST be in `noc` dir, so last path part MUST be `noc`.
- In Tower/Environments/YOURENV in `Config load preference` change all `/opt/noc` to `/usr/local/noc` (or whatever path you decided).
- Find `noc` service in Tower/Services and change path to `/usr/local/noc`.
- GOSS `tower/playbooks/NOC/system_roles/goss/defaults/main.yml` (even if you will not install `goss` service, deploy will try to create goss dir and will fail while creating `/opt/goss` on read-only root)
```shell
goss_path: "/usr/local/goss_v{{ goss_version }}"
```
## Deployment
- Enter noc control tower. - Enter noc control tower.
Open http://<IP>:8888/ in your browser. Login as admin/admin Open http://<IP>:8888/ in your browser. Login as admin/admin
- Go to environments, press "+ Create new..", enter hostname, save, then select it and "Pull". - Go to environments, press "+ Create new..", enter hostname, save, then select it and "Pull".
- Go to datacenters, press "+ Create new..", enter name, save, then select it. - Go to datacenters, press "+ Create new..", enter name, save, then select it.
- Go to nodes, create new, enter datacenter, enter type (FreeBSD), ip address, save. - Go to nodes, create new, enter datacenter, enter type (FreeBSD), ip address, save.
- Go to services, enable all services on node, save. - Go to services, enable all services on node, save.
- Go to environments again, press Deploy. - Go to environments again, press Deploy.
Do not forget to change tower's admin password Do not forget to change tower's admin password
(Upper right menu > Change Password) (Upper right menu > Change Password)
## PS: About jails # After deployment
For now there's a [bug](https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=227716) that prevents running mongodb in jail (when using mongo shell it coredumps with error ```"Failed to mlock: Resource temporarily unavailable"```), so for this moment (upcoming 12.1-RELEASE) one couldn't use FreeBSD jail for NOC. * Change `noc/etc/noc_services.conf`, FreeBSD doesn't have `taskset` and `nproc` utilities, so command for `activator-default` should be:
But to save knowledge about all other aspects about running NOC in jail besides this mongodb problem (which I think will be solved in future), here's what you need to do to run NOC in jail. ```shell
* Jail must be configured using VNET network interface, so that you will have a lo0 interface with 127.0.0.1 address on it inside a jail. IP 127.0.0.1 is sometimes hardcoded all over NOC's components, so you will have hard time deploying NOC to jail with shared network interfaces. [program:activator-default]
* Do all mentioned in [Prepare Nodes](#prepare-nodes) command = /bin/sh -c 'exec cpuset -l $((%(process_num)d %% $(/sbin/sysctl -n hw.ncpu))) ./services/activator/service.py'
* Make sure /etc/jail.conf have ```"allow.sysvipc=1"``` for PostgreSQL. ```
* During deploy there will be SSE4.2 check, which is done by greping /var/run/dmesg.boot, and this file will be empty EVERY TIME YOU START JAIL. So you have to copy host's /var/run/dmesg.boot to jail's /var/run and do deploy without restarting jail (or do this every time you restart jail). You will need this for the time of deployment only.
* If you have thinjails then probably you have read-only root in it, so you have to change ```/opt``` path to more BSD'ish ```/usr/local``` path all the way inside tower playbooks .yml files.
* GOSS tower/playbooks/NOC/system_roles/goss/defaults/main.yml
goss_path: "/usr/local/goss_v{{ goss_version }}"
* NOC tower/playbooks/NOC/noc_roles/noc/defaults/main.yml
noc_root: /usr/local/noc
* NOC tower/playbooks/NOC/noc_roles/noc/tasks/tests.yml
shell: /usr/local/noc/noc ctl status | grep -v RUNNING
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment