How to start a Web-server with nginx+uwsgi+flask

The fastest way of providing a web interface to your Python project is, in my opinion, to use Flask (http://flask.pocoo.org/). But it’s built-in web server is good only for testing and development (in my case I’ve got some strange errors like broken pipes when I tried to put in production under moderate load). So the recommended option is to use nginx and uwsgi as a web server and flash as a backend.
1) In case of a fresh installation of Ubunty 14.04 first thing you should do is to install some packages: pip for installation of usefull Python modules, python-dev in order to enable pip to install uwsgi and nginx

sudo apt-get update
sudo apt-get install python-pip python-dev nginx

2)    Let’s also install virtualenv. Ubunty Trusty comes with python 2.7.6 which lacks ssl library. My previous idea was to install python 2.7.9 into separate directory and to use it, but later I found cool molude named requests (http://docs.python-requests.org/en/latest/) which deal with all ssl stuff, so now I’d rather use systemwide python. But in order to isolate it from other admins who like to mess with python modules it’s better to use  virtualenv

sudo pip install virtualenv

3)    Now let’s create our projects directories

sudo mkdir /opt/my_flask
cd /opt/my_flask

4)    We will store our virtualized python here with a command

sudo virtualenv my_flask_env

5)    Now we have to activate it

sudo source my_flask_env /bin/activate

And the prompt would look like:

(my_flask_env)user@host:/opt my_flask$

6)    Install flask and uwsgi

sudo pip install uwsgi flask

7)    Create a sample application with flask

sudo vi /opt/my_flask/my_flask.py

Here we need to import Flask module and create a route of / which will return HTML code when requested

from flask import Flask
application = Flask(__name__)

@application.route("/")
def hello():
    return "<span>Hello world!</span>"

if __name__ == "__main__":
    application.run(host='0.0.0.0')

8)    Now we have to create uwsgi entry point, which will link uwsgi with our flaks application

sudo vi /opt/my_flask/wsgi.py
from my_flask import application

if __name__ == "__main__":
    application.run()

9)    It’s time to make a configuration file for uwsgi

sudo vi /opt/my_flask/my_flask.ini
[uwsgi]
socket = /tmp/uwsgi.sock
module = my_flask
callable = application
enable-threads = true
chmod-socket = 666
vacuum = true
die-on-term = true
req-logger = file:/tmp/reqlog
logger = file:/tmp/errlog

/tmp/uwsgi.sock – is a special socket used as a pipe between nginx and uwsgi process and uses special uwsgi protocol.

die-on-term – will help to start and stop our process with init scripts,

vacuum is needed for socket clean up

10)    Now let’s create an upstart script and a separate user

sudo useradd --no-create-home nginx
sudo vi /etc/init/my_flask.conf
description "uWSGI server of my_flask"
start on runlevel [2345]
stop on runlevel [!2345]
setuid nginx
setgid www-data
env PATH=/opt/my_flask/my_flask_env/bin
chdir /opt/my_flask
exec uwsgi --ini my_flask.ini

11)    Starting uwsgi is simple – just type

sudo start myproject

12)    Now let’s configure nginx to connect to the socket

sudo vi /etc/nginx/sites-available/default

And inside server block paste this

server {
    listen 80;
    server_name server_domain_or_IP;

    location / {
        include uwsgi_params;
        uwsgi_pass unix=:/tmp/uwsgi.sock
    }
}

13)Restart nginx with

sudo service nginx restart

And that’s it. Navigate to http://server_domain_or_IP and I’ll be able to see Hello world!
You can aslo find all logs in /tmp/reqlog and /tmp/errlog

Video broadcasting and recording with Cisco part 4. Wowza+Nginx

During these series we have installed Cisco Telepresence Content Server and Wowza Media Streaming Engine. TCS works as SIP gateway and converts SIP signalling with RTP payload to RTSP signalling with RTP for live streams. Wowza accepts RTSP and converts it to RTMP which can be used with Flash players and HLS for HTML5 (we will discuss HLS later).
Imagine that you have to provide live streaming for more than 1000 users. In that case Wowza have a solution – Wowza Origin and Wowza Edge. The idea behind it that you have one generating Wowza server and a number of retranslators (edges), which are situated closer to users. That’s great but you have to buy license for all those edges and it summs up to a big money. Maybe it’s worth investing when you have big video broadcasting events every week, but what if your weekly translations cover 700 users and such big events are held once a year. In that case there is open-source solution for retranslators: nginx web server with nginx-rtmp plugin (https://github.com/arut/nginx-rtmp-module).

1) Assume we have fresh installation of Ubuntu 14.04. Let’s install all supplementary packages.

# apt-get update
# apt-get upgrade
# apt-get install build-essential libpcre3 libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev git checkinstall

2) Create separate user for nginx

# useradd --no-create-home nginx

3) Download nginx and rtmp plugin sources

# mkdir /usr/build 
# cd /usr/build 
# git clone git://github.com/arut/nginx-rtmp-module.git 
# wget http://nginx.org/download/nginx-1.7.11.tar.gz

4) Unpack, configure, compile and install package. Again I’m using checkinstall to get deb packet and the ability to delete installed package with dpkg in future

# tar zxfv nginx-1.7.11.tar.gz
# cd nginx-1.7.11
# ./configure --prefix=/usr/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/lock/nginx.lock --with-http_ssl_module --user=nginx --group=nginx --with-http_stub_status_module --with-http_gzip_static_module --add-module=/usr/build/nginx-rtmp-module
# make
# checkinstall --pkgname nginx-rtmp

5) Create a script to launch nginx

# cd /etc/init.d
# vi nginx

Copy following to vi

# Description: starts nginx using start-stop-daemon
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/nginx/sbin:/usr/nginx/bin
DAEMON=/usr/nginx/sbin/nginx
NAME=nginx
DESC=nginx
test -x $DAEMON || exit 0
# Include nginx defaults if available
if [ -f /etc/default/nginx ] ; then
. /etc/default/nginx
fi
set -e
. /lib/lsb/init-functions
case "$1" in
start)
echo -n "Starting $DESC: "
start-stop-daemon --start --quiet --pidfile /var/run/$NAME.pid \
--exec $DAEMON -- $DAEMON_OPTS || true
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
start-stop-daemon --stop --quiet --pidfile /var/run/$NAME.pid \
--exec $DAEMON || true
echo "$NAME."
;;
restart|force-reload)
echo -n "Restarting $DESC: "
start-stop-daemon --stop --quiet --pidfile \
/var/run/$NAME.pid --exec $DAEMON || true
sleep 1
start-stop-daemon --start --quiet --pidfile \
/var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS || true
echo "$NAME."
;;
reload)
echo -n "Reloading $DESC configuration: "
start-stop-daemon --stop --signal HUP --quiet --pidfile /var/run/$NAME.pid \
--exec $DAEMON || true
echo "$NAME."
;;
status)
status_of_proc -p /var/run/$NAME.pid "$DAEMON" nginx && exit 0 || exit $?
;;
*)
N=/etc/init.d/$NAME
echo "Usage: $N {start|stop|restart|reload|force-reload|status}" >&2
exit 1
;;
esac
exit 0

5) Make it executable and add to autolaunch. Now you can start, stop and restart it with

# /etc/init.d/nginx start|stop|restart command
# chmod +x /etc/init.d/nginx
# update-rc.d -f nginx defaults

6) Now post following to /etc/nginx/nginx.conf and restart nginx

worker_processes  1;
error_log /var/log/nginx/error.log  debug;
events {
worker_connections 1024;
}
rtmp {
    server {
        listen 1935;
        application live-edge {
                live on;
                pull rtmp://<wowza IP adress>:1935 app=live;
       }
    }
}

http {
        server {
                listen 8080;
                location /stat {
                        rtmp_stat all;
                        rtmp_stat_stylesheet stat.xsl;
                }
                location /stat.xsl {
                        root /usr/build/nginx-rtmp-module/;
                }
               location /control {
                        rtmp_control all;
                }
        }
}

RTMP section of the config creates RTMP server that serves live-egde RTMP application. Whenever user opens browser with flash player, the player makes a connection to this nginx server and provides app name (live-edge) and stream number (1002). When nginx gets this request it connects to Wowza and query for app=live and same stream name. When second user comes it servres him chached data without creating additional connection to Wowza.  You can as many such retranslators as you want.

Additionally you can query nginx for connected users by consulting http://<server IP>:8080/stat.