Install FindFace Server

This section will guide you through the FindFace Server step-by-step installation process. Follow the instructions below minding the sequence.

Tip

Standalone installation can also be done from a developer-friendly installer.

In this section:

• Install FindFace Server
  • Prepare Packages for Installation
  • Licensing
  • Install Components
    • Install facenapi
    • Install nnapi
    • Install findface-upload
    • Install tntapi
      • Install tntapi standalone
      • Install tntapi cluster
  • Configure Network

Prepare Packages for Installation

FindFace Enterprise Server SDK can be installed from a local repository. You can receive the FindFace Enterprise Server SDK distributable packages from your NTechLab manager. To prepare the packages for installation, do the following:

Warning

The ntech user will be automatically created at this stage. To avoid a conflict, make sure that such a user does not already exist in the system.

1. Unpack the package with components on each designated host.

   $ sudo dpkg -i <findface_repo>.deb
   

2. Unpack the package with models. In the cluster environment, models are installed only on the findface-nnapi hosts.

   $ sudo dpkg -i <findface_data>.deb
   

3. Add a signature key on each designated host.

   $ sudo apt-key add /var/findface-repo/public.key
   

Licensing

You receive a license file from your NTechLab manager along with the FindFace Enterprise Server SDK distributable packages. If you opt for on-premise licensing, you will be also sent a Guardant USB dongle. The licensing scheme for FindFace Enterprise Server SDK is shown on the diagram below.

To provide the FindFace Enterprise Server SDK licensing, follow the steps below:

1. Install and configure the local license server NTLS.
2. If the licensable components (findface-nnapi, tntapi, fkvideo_detector and extraction-api) are installed on remote hosts, specify the NTLS host IP address in their configuration files.

To install and configure NTLS, do the following:

1. Install the NTLS component:

   $ sudo apt-get update
   $ sudo apt-get install ntls
   

   Tip

   In the NTLS configuration file, you can change the license folder and specify your proxy server IP address if necessary. You can also change the NTLS web interface remote access settings. To open the NTLS configuration file, execute:

   $ sudo vi /etc/ntls.cfg
   

   If necessary, change the license folder in the license-dir parameter. By default, license files are stored at /ntech/license:

   $ license-dir = /ntech/license
   

   If necessary, uncomment the proxy line and specify your proxy server IP address:

   proxy = http://192.168.1.1:12345
   

   By default, you can access the NTLS web interface from any remote host (ui = 0.0.0.0:3185). To indicate that accessing the NTLS web interface must originate from a specific IP address, edit the ui parameter:

   ui = 127.0.0.1:3185
   

2. Enable the NTLS service autostart and lauch the service:

   $ sudo systemctl enable ntls && sudo systemctl start ntls
   

3. Upload the license file via the NTLS web interface http://<NTLS_IP_address>:3185/#/. You can also use the NTLS web interface to consult your license information, and upgrade or extend the license.

4. For on-premise licensing, insert the Guardant dongle into a USB port.

Important

If the licensable components (findface-nnapi, tntapi, fkvideo_detector and extraction-api) are to be installed on remote hosts, keep in mind that you have to specify the IP address of the NTLS host in their configuration files after installation.

Install Components

Now that you have prepared the FindFace Enterprise Server SDK packages and provided the licensing, install the Server components on designated host(s) according to your architecture outline.

Install facenapi

Install and configure the findface-facenapi component as follows:

1. Install the component.

   $ sudo apt-get update
   $ sudo apt-get install python3-facenapi
   

2. If MongoDB is installed on a remote host, specify its IP address in the findface-facenapi configuration file.

   ## Open the configuration file.
   $ sudo vi /etc/findface-facenapi.ini
   
   ## Specify the MongoDB host IP address.
   mongo_host = '192.168.113.1'
   

3. Check if the component is runnable. To do so, invoke the findface-facenapi application by executing the command below. As the application is invoked, hold 1 minute, and if no errors display, hit Ctrl+C.

   ## If MongoDB is installed on the same host
   $ findface-facenapi
   
   ## If MongoDB is installed on a remote host
   $ sudo findface-facenapi --config=/etc/findface-facenapi.ini
   

4. Check if the findface-facenapi service autostart at system startup is enabled.

   $ systemctl list-unit-files | grep findface-facenapi
   
   ## You should see the following output
   findface-facenapi.service disabled
   

5. Enable the service autostart and launch the service.

   $ sudo systemctl enable findface-facenapi.service && sudo service findface-facenapi start
   

6. Make sure that the service is up and running.

   $ sudo service findface-facenapi status
   
   ## The command will return a service description, a status (should be Active), path and running time.
   

Tip

You can view the findface-facenapi logs by executing:

$ sudo tail -f /var/log/syslog | grep facenapi

Install nnapi

Install and configure the findface-nnapi component as follows:

Tip

You may also want to learn about gender, age and emotions recognition.

1. Install the component.

   $ sudo apt-get update
   $ sudo apt-get install findface-nnapi
   

2. If NTLS is installed on a remote host, specify its IP address in the findface-nnapi configuration file.

   $ sudo vi /etc/findface-nnapi.ini
   
   ## Specify the NTLS IP address:
   license_ntls_server = 192.168.113.2:3133
   

3. Check if the component is runnable. To do so, invoke the findface-nnapi application by executing the command below. As the application is invoked, hold 1 minute, and if no errors display, hit Ctrl+C.

   $ findface-nnapi
   

4. Check if the findface-nnapi service autostart at system startup is enabled.

   $ systemctl list-unit-files | grep findface-nnapi
   
   ## You should see the following output
   findface-nnapi.service  disabled
   

5. Enable the service autostart and launch the service.

   $ sudo systemctl enable findface-nnapi.service && sudo service findface-nnapi start
   

6. Make sure that the service is up and running.

   $ sudo service findface-nnapi status
   
   ## The command will return a service description, a status (should be Active), path and running time.
   

Tip

You can view the findface-nnapi logs by executing:

$ sudo tail -f /var/log/syslog | grep nnapi

Install findface-upload

To store all original images which have been processed by Server, as well as such Server artifacts as face thumbnails and normalized images, install and configure the findface-upload component.

Tip

Skip this procedure if you do not want to store these data on the FindFace Enterprise Server SDK host. In this case, only face features vectors (facens) will be stored (in MongoDB and Tarantool databases).

Do the following:

1. Install the component:

   $ sudo apt-get update
   $ sudo apt-get install findface-upload
   

2. By default the original images, thumbnails and normalized images are stored at /var/lib/ffupload/uploads/. You can view this folder content at http://127.0.0.1:3333/uploads/ in your browser. Make sure that this address is available. You will have to specify it when configuring network.

   $ curl -I http://127.0.0.1:3333/uploads/
   ## The command should return the following message.
   HTTP/1.1 200 OK
   

Install tntapi

The tntapi component connects the Tarantool database and the facenapi component, transferring search results from the database to facenapi for further processing. To increase search speed, multiple tntapi shards can be created on each Tarantool host. Their running concurrently leads to a remarkable increase in performance. Each shard can handle up to approximately 10,000,000 faces. In the case of standalone deployment, you need only one shard (created by default), while in a cluster environment the number of shards has to be calculated depending on several parameters (see below).

Install tntapi standalone

Install and configure the tntapi component as follows:

1. Install tntapi. Tarantool will be installed automatically along with tntapi.

   $ sudo apt-get update
   $ sudo apt-get install findface-tarantool-server
   

2. Disable the tarantool example service autostart and stop the service.

   $ sudo systemctl disable tarantool@example && sudo systemctl stop tarantool@example
   

3. For a small-scale project, the tntapi shard created by default (tarantool@FindFace) would suffice as 1 shard can handle up to 10,000,000 faces. Configuration settings of the default shard are defined in the file /etc/tarantool/instances.enabled/FindFace.lua. We strongly recommend you not to add or edit anything in this file, except the maximum memory usage (memtx_memory), the NTLS IP address required for the tntapi licensing, and the remote access setting. The maximum memory usage should be set in bytes, depending on the number of faces the shard handles, at the rate roughly 1280 byte per face.

   ## Open the configuration file
   $ sudo vi /etc/tarantool/instances.enabled/FindFace.lua
   
   ## Edit the value due to the number of faces the shard handles. The value ``1.2*1024*1024*1024`` corresponds to 1,000,000 faces.
   memtx_memory = 1.2 * 1024 * 1024 * 1024,
   
   ## Specify the NTLS IP address if NTLS is remote.
   FindFace.start(“127.0.0.1”, 8001, {license_ntls_server=“192.168.113.2:3133”})
   
   ## With standalone deployment, you can access Tarantool by default only locally (127.0.0.1). If you want to access Tarantool from a remote host, either specify the remote host IP address in the FindFace.start section, or change ``127.0.0.1`` to ``0.0.0.0`` there to allow access to Tarantool from any IP address.
   ## Remote host
   FindFace.start("192.168.113.10", 8001, {license_ntls_server=“192.168.113.2:3133”})
   ## Allow access from any IP address
   FindFace.start("0.0.0.0", 8001, {license_ntls_server=“192.168.113.2:3133”})
   

4. Configure the tntapi shard to autostart and start the shard.

   $ sudo systemctl enable tarantool@FindFace && sudo systemctl start tarantool@FindFace
   

5. Retrieve the shard status.

   $ sudo systemctl status tarantool@FindFace
   ## The command will return a service description, a status (should be Active), path and running time.
   

   Tip

   You can view the tntapi logs by executing:

   $ sudo tail -f /var/log/tarantool/FindFace.log
   

6. The tntapi.json file containing the tntapi shard parameters is automatically installed along with tntapi into the /etc/ folder. You will have to uncomment the path to tntapi.json when configuring network.

Install tntapi cluster

Install and configure the tntapi component as follows:

1. Install tntapi on designated hosts. Tarantool will be installed automatically along with tntapi.

   $ sudo apt-get update
   $ sudo apt-get install findface-tarantool-server
   

2. Create tntapi shards on each tntapi host. To learn how to shard, let’s consider the example where a cluster environment contains 4 database hosts with 4 shards on each to be created.

   Important

   When creating shards in large installations, observe the following rules:

   1. tntapi shard can handle approximately 10,000,000 faces.
   2. The number of shards on a single host should not exceed the number of its physical processor cores minus 1.

3. Disable the tarantool example service autostart and stop the service. Do so for all the 4 hosts.

   $ sudo systemctl disable tarantool@example && sudo systemctl stop tarantool@example
   

4. Disable the shard created by default. Do so for all the 4 hosts.

   $ sudo systemctl disable tarantool@FindFace
   

5. Write a bash script shard.sh that will automatically create configuration files for all shards on a particular host. Do so for the 4 hosts. Use the following script as a base for your own code. The exemplary script creates 4 shards listening to the ports: tntapi 33001..33004 and http 8001..8004.

   Important

   The script below creates configuration files based on the default configuration settings stored in the file /etc/tarantool/instances.enabled/FindFace.lua. We strongly recommend you not to add or edit anything in the default file, except the maximum memory usage (memtx_memory) and the NTLS IP address required for the tntapi licensing. The maximum memory usage should be set in bytes for each shard, depending on the number of faces a shard handles, at the rate roughly 1280 byte per face.

   ## Open the configuration file
   $ sudo vi /etc/tarantool/instances.enabled/FindFace.lua
   
   ## Edit the value due the number of faces a shard handles. The value ``1.2*1024*1024*1024`` corresponds to 1,000,000 faces.
   memtx_memory = 1.2*1024*1024*1024,
   
   ## Specify the NTLS IP address if NTLS is remote.
   FindFace.start(“127.0.0.1”, 8001, {license_ntls_server=“192.168.113.2:3133”})
   

   #!/bin/sh
   set -e
   
   for I in `seq 1 4`; do
          TNT_PORT=$((33000+$I)) &&
          HTTP_PORT=$((8000+$I)) &&
          sed "
                  s#/opt/ntech/var/lib/tarantool/default#/opt/ntech/var/lib/tarantool/shard_$I#g;
                  s/listen = .*$/listen = '127.0.0.1:$TNT_PORT',/;
                  s/\"127.0.0.1\", 8001,/\"0.0.0.0\", $HTTP_PORT,/;
          " /etc/tarantool/instances.enabled/FindFace.lua > /etc/tarantool/instances.enabled/FindFace_shard_$I.lua;
   
          mkdir -p /opt/ntech/var/lib/tarantool/shard_$I/snapshots
          mkdir -p /opt/ntech/var/lib/tarantool/shard_$I/xlogs
          mkdir -p /opt/ntech/var/lib/tarantool/shard_$I/index
          chown -R tarantool:tarantool /opt/ntech/var/lib/tarantool/shard_$I
          echo "Shard #$I inited"
   done;
   

   Tip

   Download the exemplary script.

6. Run the script from the home directory.

   $ sudo sh ~/shard.sh
   

7. Check the configuration files created.

   $ ls /etc/tarantool/instances.enabled/
   
   ## You should get the following output
   example.lua FindFace.lua FindFace_shard_1.lua FindFace_shard_2.lua FindFace_shard_3.lua FindFace_shard_4.lua
   

8. Launch all the 4 shards. Do so on each host.

   $ for I in `seq 1 4`; do sudo systemctl enable tarantool@FindFace_shard_$I; done;
   $ for I in `seq 1 4`; do sudo systemctl start tarantool@FindFace_shard_$I; done;
   

9. Retrieve the shards status.

   $ sudo systemctl status tarantool@FindFace*
   
   ## You should get the following output:
   tarantool@FindFace_shard_3.service - Tarantool Database Server
   Loaded: loaded (/lib/systemd/system/tarantool@.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-10 16:22:07 MSK; 32s ago
   ...
   tarantool@FindFace_shard_2.service - Tarantool Database Server
   Loaded: loaded (/lib/systemd/system/tarantool@.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-10 16:22:07 MSK; 32s ago
   ...
   tarantool@FindFace_shard_1.service - Tarantool Database Server
   Loaded: loaded (/lib/systemd/system/tarantool@.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-10 16:22:07 MSK; 32s ago
   ...
   tarantool@FindFace_shard_4.service - Tarantool Database Server
   Loaded: loaded (/lib/systemd/system/tarantool@.service; disabled; vendor preset: enabled)
   Active: active (running) since Tue 2017-01-10 16:22:07 MSK; 32s ago
   ...
   

   Tip

   You can view the tntapi logs by executing:

   $ sudo tail -f /var/log/tarantool/FindFace_shard_{1,2,3,4}.log
   

10. On the findface-facenapi host, create a file tntapi_cluster.json containing the addresses and ports of all the shards. Distribute available shards evenly over ~1024 cells in one line. Click here to see the file for 4 hosts with 4 shards on each.

    Tip

    You can create tntapi_cluster.json as follows:

    1. Create a file that lists all the shards, each shard with a new line (click here to view the example).

       $ sudo vi s.txt
       

    2. Run the script below (click here to view the script). As a result, a new file tntapi_cluster.json will be created and contain a list of all shards distributed evenly over 1024 cells.

    cat s.txt | perl -lane 'push(@s,$_); END{$m=1024; $t=scalar @s;for($i=0;$i<$m;$i++){$k=int($i*$t/$m); push(@r,"\"".$s[$k]."\"")} print "[[".join(", ",@r)."]]"; }' > tntapi_cluster.json
    

11. Move tntapi_cluster.json to the directory /etc/. You will have to uncomment and specify the path to tntapi_cluster.json when configuring network.

Configure Network

After you install the FindFace Server components, configure their interaction with each other. Do the following:

1. Open the findface-facenapi.ini configuration file:

   $ sudo vi /etc/findface-facenapi.ini
   

2. Uncomment and/or edit the settings to align with your network specifications, substituting the suggested values with actual location:

   ffupload_url = 'http://127.0.0.1:3333'
   mongo_host = '127.0.0.1'
   nnapi_url = 'http://127.0.0.1:18088'
   tntapi_servers_file = '/etc/tntapi.json'
   

   Warning

   The findface-facenapi.ini content must be correct Python code.

   Note

   Do not specify ffupload_url if the findface-upload component is not installed.

3. By default, if one or several tntapi shards are out of service during face identification, findface-facenapi returns an error. If necessary, uncomment the tntapi_ignore_search_error parameter and assign it True. In this case findface-facenapi will use available tntapi shards to obtain face identification results, indicating the number of available servers vs the total number of servers in the response:

   tntapi_ignore_search_errors = True
   

4. Restart all the FindFace Enterprise Server SDK services and nginx (for findface-upload) on the relevant host(s).

   $ sudo service 'findface*' restart
   $ sudo service nginx restart
   

5. Check the services status. The command will return the services description, status (should be Active), path and running time.

   $ sudo service 'findface*' status
   $ sudo service nginx status