Using FastAPI to deploy Machine Learning models

Use FastAPI and Docker to deploy a classification Machine Learning model built in Scikit-Learn

This tutorial follows on the previous tutorial: Serve your first model with Scikit-Learn + Flask + Docker: 👇

Serve your first model with Scikit-Learn + Flask + Docker

In this tutorial, we will implement the same microservice to serve a machine learning model for classification built on Scikit-Learn but using FastAPI instead of Flask. Why? Well, FastAPI is a modern, fast (high-performance) and relevant framework for building web APIs with Python, a good alternative to Flask, and has gained popularity in recent years. Created by Sebastián Ramírez back in 2018 is usually praised by its superb documentation and great design.

You can check the full FastAPI documentation here: 👇

GitHub - tiangolo/fastapi: FastAPI framework, high performance, easy to learn, fast to code, ready…

FastAPI compares favorably to Flask on several aspects. Flask does not provide data validation, automatic documentation or asynchronous capabilities (without any extension). FastAPI also tends to be consistently faster than Flask as presented by Christopher GS in “FastAPI vs Flask — The Complete Guide” [1], by using ASGI (Asynchronous Server Gateway Interface) instead of WSGI:

Web Frameworks Benchmark

In this tutorial we will create a Docker image for deploying a model using FastAPI as part of the stack, you can find the full code and documentation here: 👇

GitHub - cwallaceh/sklearn_fastapi_docker: Deployment of ML models using Python's Scikit-Learn +…

Setting up the Environment

First we now need to set up our working environment, for this, it is useful to start by creating a virtual environment for the project. This is an important step because, by default, every Python project on your system will use the same directories to store and retrieve site-packages. By creating a virtual environment we help to create an isolated environment for Python projects, to keep track of every package. This way each project can have its own dependencies, regardless of what dependencies every other project has.

Install the package needed for creating a virtual environment*:

$ pip install virtualenv

*If you are using Python 3, then you should already have the virtualenv module from the standard library installed.

Next up, create a new virtual environment with the name venv:

$ virtualenv venv

It’s also helpful to specify the Python version used to create the environment by using the option -p and the Python path (This is helpful later when building the Docker image to match the same Python version).

$ virtualenv -p path_to_python venv

Finally, we can activate the virtual environment:

$ source venv/bin/activate

Commands are a bit different if you are working with Windows, however, this is a really great guide to get going with virtual environments in Windows:

https://medium.com/co-learning-lounge/create-virtual-environment-python-windows-2021-d947c3a3ca78

Python packages

Once we have the virtual environment activated we need to install the software dependencies (site-packages), a list of the Python packages and their versions are listed in the file requirements.txt:

We can install the dependencies by running the command:

$ pip install -r requirements.txt

After the installation is done, we can check we have correctly installed all the packages in our virtual environment:

$ pip freeze

We should see the same list in the file requirements.txt, this means that now we are ready to train our model.

After setting up the virtual environment, installing the required Python packages, we can move forward to training the model and building the application. But now let’s look at the project structure:

(root) sklearn_fastapi_docker/
├── code/
│   └── train.py
├── data/
│   └── breast_cancer.csv
├── model/
│   └── model_binary.dat.gz
├── ms/
│   ├── __init__.py
│   └── functions.py
├── tests/
│   └── example_calls.txt
├── .gitignore
├── Dockerfile
├── LICENSE
├── README.md
├── main.py
└── requirements.txt

In the folder code/, we will find the main script for training our model train.py, here we basically follow the standard data science process for training a model: loading the data, preprocessing, training and finally saving the model inside the folder model/.

$ python code/train.py

Web application

Now we are ready to create our FastAPI application!

We will start by creating the web application using FastAPI following the script in main.py:

Here we create the routes for the endpoints of our application, an /info route with the GET method to retrieve the information from the model and a /predict route with the POST method to receive input data, run the data through the pipeline and produce a prediction for the client. The file ms/functions.py contains the auxiliary functions in helping run the model:

At the beginning of the post we mentioned that one of the advantages of using FastAPI is data validation. FastAPI combines Python type hints and the Pydantic package to add this behavior. In the ‘Input’ and ‘Output’ classes declared in main.py we provided the expected variables, types and additional validations and constraints for each input and output of the model, which is useful in the case of ML ensuring correct operation of the machine learning model!

We can now go ahead and test our application by running:

$ uvicorn main:app

This command will run the development server locally and listen to port 8000 where we can test our application! The folder /tests contain some example calls to test that our application is up and running:

$ curl -X GET http://localhost:8000/info

The service should respond:

{
  "name": "Breast Cancer Wisconsin (Diagnostic)",
  "version": "v1.0.0"
}

FastAPI provide some additional amazing features such a automatic documentation, with the application running we can visit the endpoint /docs for the swagger documentation of our recently created API, and /redoc for redocly documentation:

Swagger documentation with endpoints and validations

Redoc documentation with endpoints and validations

Container

Finally, we can create our container, following the Dockerfile we give the instructions to create an image with everything we need to run the application we just created.

An image is essentially built from the instructions for a complete and executable version of an application, which relies on the host OS kernel. We start from the official base image of python:3.10, copy our application files (including the app, the model, and the requirements), install the requirements, and run the application by using uvicorn exposed through the port 8000.

We are ready to build the image by running:

$ docker build . -t sklearn_fastapi_docker

The process should take a couple of minutes, but in the end, we are going to have a Docker image with the name sklearn_fastapi_docker which will serve as the base for our application.

Serving and Testing

We are now ready to serve our containerized application as a service! We can test it locally by running:

$ docker run -p 8000:8000 sklearn_fastapi_docker

This will launch the application at localhost where we will be able to test it on the port 8000, we can use another example call to test it with a single observation:

curl -H "Content-Type: application/json" -d '{
  "concavity_mean": 0.3001,
  "concave_points_mean": 0.1471,
  "perimeter_se": 8.589,
  "area_se": 153.4,
  "texture_worst": 17.33,
  "area_worst": 2019.0
}' -XPOST http://0.0.0.0:8000/predict

The application should respond:

{
  "label": "M",
  "prediction": 1
}

And we are done! Our application is ready to be served as a prediction service!

I must add that working with FastAPI feels very easy for creating this type of applications for machine learning models. Data validation and documentation provide two benefits of working with the tool without the need for additional components. Give it a try!

References

[1] Christopher Samiullah, 2021 “FastAPI vs Flask — The Complete Guide” https://christophergs.com/python/2021/06/16/python-flask-fastapi/