Serving something useful
Connect the domain to the server.
If someone points their web browser to your custom domain, what would happen? Right now, the only thing would probably be a landing page from your registrar. You’ll need to make sure that the domain actually points to the IP address of your Digital Ocean server. Once it does, we can configure the server to actually, you know… serve something.
Digital Ocean makes it pretty straight forward with their custom DNS servers. First, go to your namecheap account, find the domain you bought, and click manage. We want to then change the nameservers to Custom DNS, pointing to DO’s nameservers. At the time of publication, these would be ns1.digitalocean.com, ns2.digitalocean.com, and ns3.digitalocean.com. Check out this tutorial for up-to-date DNS addresses or instructions for other registrars.
Then, we need to configure our Digital Ocean account. Check out this tutorial for details. Sign in and click on Networking on the left pane. Enter the domain for your project. If you have multiple droplets, make sure you’re adding it to the correct DO project. Then make sure there’s an A record where the hostname is $CUSTOM_DOMAIN.$TLD (TLD = top level domain, like .com or .xyz) and that it directs to your droplet’s IP address. You will most likely want to add a CNAME record where the hostname is www.$CUSTOM_DOMAIN.$TLD that is an alias of $CUSTOM_DOMAIN.$TLD. Now if a web brower is searching for your website, it should end up at the IP address of your droplet! I'm skipping this step so my ESP32 keeps pushing to the existing server. I'll come back and change this when everything is ready on the 20.04 droplet.
Configure a test NGINX webserver
Of course your droplet won’t do much with an HTTP GET request from a browser yet. We are going to use NGINX to serve our website. Let’s try out the simplest option, based on this tutorial. First install nginx using apt – our firewall should already be configured OK. Check that the nginx server is running using
systemctl status nginx
If it’s running, then browsing to your server’s IP address should display the nginx test page. Fantastic! Now, we set up the server block as recommended in step 5 of the tutorial. Read that section and follow it carefully. We will need to modify the contents of server block later; we will use it as a map to hand off requests to one service or another depending on whether someone is trying to look at the graph at our data or trying to update the weather collection database. Now, we should be able to view the example HTML file created in the tutorial if we access the website through the server IP address or the website’s custom domain using an HTTP request.
Get a certificate from Let’s Encrypt and make sure https addresses work.
We want to enable https for our server. To do that, we’ll use another tutorial. Again, our firewall should still be configured to allow the correct traffic (this time, on port 443 for https). Certbot makes it relatively easy to manage the Let’s Encrypt certificates. Following the tutorial should be no problem; verify that your browser doesn’t have any problem seeing the test page using https.
Get NGINX to serve a test site using flask.
Because we’re eventually going to plot data using python, we will also serve the website using a python framework: flask, the simplest one. Let’s make sure we can get flask working with nginx, rather than serving the static html test page we’ve been responding with so far.
Yet again, our incremental step has us following another Digital Ocean tutorial. Hopefully, these steps are small enough and the tutorials verbose enough that there aren’t too many knowledge gaps being leapt over. This tutorial introduces a new concept that we’ll make use of several times in the future: Python virtual environments. You should make a folder to serve as a home base for the python code in this virtual environment, and then create a new environment using
python3 -m venv $PROJECT_NAME_ENV #use a logical name here: match it with the folder structure, and make sure that it's as descriptive as possible to what its end use will be.
You enter a virtual environment using
source $PROJECT_NAME_ENV/bin/activate
which you can confirm by looking at the domain prefix in your terminal. You can leave this as a test environment/process, or name it something like SQL_API -- that's what it'll become in the next post in this series.
Now that you’re inside the virtual environment, which you specified to be python 3 when creating it, all the python-related commands that are appended with 3 in ubuntu don’t need it any more – i.e., instead of using pip3 to install the python3 version of a package, you install it using pip. Confusing? You get used to it… after troubleshooting things.
You can follow the tutorial to make a simple python app.
Then, you’ll have to get uwsgi working. I like to think of nginx+uwsgi as a translater between the internet (http) world, and our own server’s python world. Once we have the uwsgi.ini file set up If you ever need to troubleshoot a web service, I recommend changing the number of workers to 1 by making the line 'processes = 1'. This will ensure that you're always interacting with the same worker as you send + receive test data. Otherwise, you are not guaranteed to be interacting with the same process, and they don't share memory: if you are storing and recalling data, it will act weird if you are alternating between processes. This behavior took me days to troubleshoot when testing a simple memory-based database storage/retrieval REST API. , we can configure a system service to start our virtual environment + uwsgi + python file so the server automatically is ready to handle incoming requests. Once we set things up as a service, we can treat it like other system services (i.e., use systemctl and journalctl commands to control and troubleshoot). This requires making a .service file in the /etc/systemd/system/ folder and then running
sudo systemctl start $SERVICE_NAME
sudo systemctl enable $SERVICE_NAME #will start on boot
The last thing the tutorial does is modify the nginx configuration file to forward requests on your website to the correct system socket (which will always be up and running, now that we’ve configured it as a service). After following along to the end of the tutorial, check that you can now see the test flask website using http and https.
Relax
Give yourself a pat on the back: the nginx + uwsgi + flask (+arbitrary python code) + systemd process is a framework that can be used for nearly everything you’d want to do related to the internet. And now, you know how to set it up from scratch!