Building a Web App From the Ground Up: Django Rest Framework API Development — Part 3

 In Software Engineering, Tutorials

This post is part of a series of mine on building web applications from the ground up. It covers everything you need to know to completely finish a web application. If you haven’t been following the series, please go back and read part 1

Recap

In the last part of this series we successfully setup our API project. We made a Git repository, initialized our overarching project files, and developed a small amount of the API. Now we can continue to develop the models and views in our app.

Create a new Django App

In the last part of the series, we created a test endpoint under the main project, but we really should create a new Django app for related models and views.

But before we get too far, why don’t we delete the test_view.py file we made in the last part and the references to it in urls.py.

Now, create a new Django app with the following command.

python manage.py startapp todo

Be sure to add the “todo” app to the INSTALLED_APPS list in settings.py.

Creating Models

Now that we have a shiny new app to play with, let’s make some models for it.

Some of you might still be wondering, what the **** is a model? Simply put, a model helps Django map Python objects to database tables.

Since we’re building a ToDo app, I think we should start by making a ToDo list. But what composes a ToDo list? Well, there’s the list and uh… list items.

When we created a new app, it should have made a file located at src/todo/models.py. Edit that file to add our models.

# import the django models module
from django.db import models

# This class will more or less map to a table in the database
class ToDoList(models.Model): 
    """ A Model for representing single line items in a ToDo List. """
    # defines a required text field of 100 characters. 
    title = models.CharField(max_length=100)

# This class will more or less map to a table in the database
class ToDoListItem(models.Model):
    """ A Model for representing single line items in a ToDo List. """
    # Define a foreign key relating this model to the todo list model.
    # The parent will be able to access it's items with the related_name
    # 'items'. When a parent is deleted, this will be deleted as well. 
    todo_list = models.ForeignKey(
        'ToDoList',
        related_name='items', on_delete=models.CASCADE
    )
    # this defines a required title that cannot be more than 100 characters.
    title = models.CharField(max_length=100)
    # this defines an arbitrary length text field which is optional.
    description = models.TextField(blank=True, default='')

Now that the models are defined, we have to do a migration in Django. You can do this with the following commands.

python manage.py makemigrations todo
python manage.py migrate

This will generate SQL files that are then run on the database to apply the changes you made to your model. Afterwords, your ToDo models should now be able to be used in the application.

Test Your Models

Writing tests is always a good practice. However, I’ll be the first to admit that I don’t always write tests for every model in my Django app. But today we will!

When we created our app, it made a file, src/todo/test.py. However, if you write a lot of tests, this can easily get messy. Trust me, I know. So before we start writing tests, let’s just delete this file, and create a new folder named test in the todo/ folder. Within that folder, create an empty file named __init__.py. As a reminder, this is how Python recognizes the test/ folder as a package.

Now, make a new file src/todo/test/test_models.py file and add the following.

from django.test import TestCase
from todo.models import ToDoList, ToDoListItem

class ToDoListTestCase(TestCase):
    """Tests for the todo list models"""

    def test_todo_list(self): 
        """Test creating a todo list with no items"""
        # create a todo list object (this saves it to the database)
        ToDoList.objects.create(title="Test List")
        # verify the object was saved
        todo_list = ToDoList.objects.get(title="Test List")
        # verify the data is accurate and exists
        self.assertEqual(todo_list.title, "Test List")
        # delete the list 
        todo_list.delete()
        # try to retrieve the todo list and make sure it's not there
        try: 
            retrieved_list = ToDoList.objects.get(title="Test List")
        # catch the DoesNotExist error that we're hoping to get
        except ToDoList.DoesNotExist: 
            retrieved_list = None
        self.assertEqual(retrieved_list, None)

This is a good basic test for a ToDoList. But we aren’t testing items at all here. Let’s add a quick test to verify items function how we expect them to.

def test_todo_list_items(self):
    """
    This test creates a todo list, adds an item to it, and verifies
    that item is accessible through the 'items' related name. Then,
    it deletes everything.
    """ 
    todo_list = ToDoList.objects.create(title="Test") 
    todo_list_item = ToDoListItem.objects.create(
        todo_list=todo_list,
        title="Test Item",
        description="This is a test todo list item")

    # verify that the related name returns the todo_list item
    self.assertEqual(todo_list.items.count(), 1)
    self.assertEqual(todo_list.items.first(), todo_list_item)

    # now delete the list. This should delete any items with it
    todo_list.delete()
    # verify the todo list item was deleted with the list
    # due to the CASCADE attribute we gave our model
    try:
        retrieved_item = ToDoListItem.objects.get(title="Test Item")
    except ToDoListItem.DoesNotExist:
        retrieved_item = None
    self.assertEqual(retrieved_item, None)

Now that the tests are created, you can run them with the following command.

python manage.py test

Create a Serializer

Serializers help transform data from your models into JSON, which is returned by the API. Conversely, it also helps to parse data POSTED to your API and create new models.

Create a new file src/todo/serializers.py.

# import the django rest framework serializers
from rest_framework import serializers
# import our models
from todo import models

class ToDoListItem(serializers.ModelSerializer):
    """
    Create the first serializer for ToDoListItems
    This is a model serializer, so it will autogenerate a
    full serializer from the model.
    """
    class Meta:
        # specify the model to use
        model = models.ToDoListItem
        # specify the field names we want to show in the api
        fields = ('id', 'todo_list_id', 'title', 'description')

class ToDoList(serializers.ModelSerializer):
    """
    Create a second serializer for the ToDoList model.
    This will have the above serializer nested within it.
    """
    items = ToDoListItem(many=True)

    class Meta:
        # specify the ToDoList model
        model = models.ToDoList
        # specify the fields from ToDoList that we want our API to return/consume
        fields = ('id', 'title', 'items')

    def create(self, validated_data):
        """
        Override the default create method so that we can serialize
        whole ToDoList objects with ToDoListItems.

        :param validated_data: A Dictionary of values to use for object creation.
        :type validated_data: OrderedDict
        :returns ToDoList: A ToDoList model object
        """
        # remove the value of items from the ToDoList validated data. We'll use this later
        items_data = validated_data.pop('items')
        # create a new ToDoList with the validated data passed in
        todo_list = models.ToDoList.objects.create(**validated_data)
        # for each item in the 'items' validated data
        for item_data in items_data:
            # modify it's validated data to reference the ToDoList we just made
            item_data['todo_list_id'] = todo_list.id
            # Create the ToDoListItem with the item data
            models.ToDoListItem.objects.create(**item_data)
        # after this return the todo_list we made
        return todo_list

Testing your Serializers

Moving on, we now have a serializer created. And we think it works. But to be sure, we should write a test for it.

Create a new test file at test/test_serializers.py with the following contents.

from django.test import TestCase
from todo import serializers

class ToDoListTestCase(TestCase):
    """Tests for the todo list models"""

    def test_todo_list_create(self):
        """
        Define some json data we expect to receive and use the 
        serializer to parse it into models. Test the models
        to make sure they're correct.
        """
        # define data that resembles what would come through the API.
        # A Python dictionary is often what JSON data is parsed into initially.
        data = {
            'title': 'Test List',
            'items': [
                {
                    'title': 'Test Item 1',
                    'description': 'This is test item 1'
                },
                {
                    'title': 'Test Item 2',
                    'description': 'This is test item 2'
                }
            ]
        }

        # pass the data into the serializer to try and parse it
        serializer = serializers.ToDoList(data=data)
        # verify that the serializer thinks the data is valid
        self.assertTrue(serializer.is_valid())

        # get the object parsed from the serializer
        todo_list = serializer.save()

        # verify the title is correct
        self.assertEqual(todo_list.title, 'Test List')
        # verify it has two items
        self.assertEqual(todo_list.items.count(), 2)

This will feed data into our serializers and make sure that it parses it correctly. As an additional excersize, try writing some tests that prove invalid data is not parsed correctly.

Run the tests again with the following command.

python manage.py test

Creating Views

Django views define what API endpoints will do. These API endpoints will either get data, post new data, update existing data, or delete data.

In views.py, add the following views which will allow us to get and create todo lists.

from django.shortcuts import render
from django.http import Http404
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework import status

from todo import serializers, models

class ToDoList(APIView):
	"""
	List all todo lists when the GET method is called. 
	Add new ToDo lists when the POST method is called.
	Delete a ToDo list when the DELETE method is called.
	"""

	def get(self, request, format=None):
		"""
		Return a list of all the ToDoList objects in the database.
		"""
		# query the database for all instances of the ToDoList model
		todo_lists = models.ToDoList.objects.all()

		# serialize the data into a returnable format
		serializer = serializers.ToDoList(todo_lists, many=True)

		return Response(serializer.data)

	def post(self, request, format=None):
		"""
		Creates a new ToDoList with the given request data.
		"""
		# deserialize the data from the request
		serializer = serializers.ToDoList(data=request.data)
		# if the data validation done by the serialize passes
		if serializer.is_valid():
			# then save the ToDoList to the database and return the resulting ToDoList
			serializer.save()
			return Response(serializer.data)
		else:
			# Throw a 400 error if the serializer detected the data wasn't valid
			return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

	def delete(self, request, todo_list_id, format=None):
		"""Deletes a ToDoList from the database"""
		# do this in a try block to catch errors if todo_list_id isn't real
		try:
			# if the list is real then delete it
			todo_list = models.ToDoList.objects.get(id=todo_list_id)
			todo_list.delete()
			return Response(status=status.HTTP_204_NO_CONTENT)
		# catch the DoesNotExist error and return an Http error
		except models.ToDoList.DoesNotExist:
			raise Http404

class ToDoListItem(APIView):
	"""
	Endpoints for adding/deleting individual ToDoListItems.
	"""

	def post(self, request, todo_list_id, format=None):
		"""Add new ToDoList items to an existing ToDoList"""
		# do this in a try block in case the todo_list_id is not real
		try:
			# Query the db for the existing list
			todo_list = models.ToDoList.objects.get(id=todo_list_id)
			# deserialize the data from the request
			serializer = serializers.ToDoListItem(data=request.data)

			# if the ToDoListItem is valid then add the list id and save it
			if serializer.is_valid():
				serializer.validated_data['todo_list_id'] = todo_list_id
				serializer.save()
				return Response(serializer.data)

			else:
				# return an HTTP 400 error if the data was not correct
				return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

		# ALWAYS catch your DoesNotExist errors :)
		except models.ToDoList.DoesNotExist:
			raise Http404

	def delete(self, request, todo_list_item_id, format=None):
		"""Deletes a ToDoListItem from the database"""
		# try to find the ToDoListItem with the id
		try:
			item = models.ToDoListItem.objects.get(id=todo_list_item_id)
			# if the item exists then delete it
			item.delete()
			return Response(status=status.HTTP_204_NO_CONTENT)
		# return the 404 error if it doesn't exist
		except models.ToDoListItem.DoesNotExist:
			return Http404

Now we have several views defined. Unfortunately, these don’t really do a heck of a lot unless we map URLs to them. Open your urls.py file and change it so it looks like the following:

from django.conf.urls import url
from rest_framework.urlpatterns import format_suffix_patterns
from todo import views

urlpatterns = [
    # Maps to ToDoList.get and ToDoList.post
    url(r'^todo_lists$', views.ToDoList.as_view()),
    # Maps to ToDoList.delete
    url(r'^todo_lists/(?P<todo_list_id>[0-9]+)$', views.ToDoList.as_view()),
    # maps to ToDoListItem.post
    url(r'^todo_lists/(?P<todo_list_id>[0-9]+)/items$', views.ToDoListItem.as_view()),
    # maps to ToDoListItem.delete
    url(r'^todo_lists/items/(?P<todo_list_item_id>[0-9]+)$', views.ToDoListItem.as_view()),
]

urlpatterns = format_suffix_patterns(urlpatterns)

This maps specific URL patterns to the different views. For example, now if I make the request GET localhost:8000/todo_lists, I will receive a list of all the ToDoLists. But don’t take my word for it, let’s test it out!

Testing your Views

Now, lets write some basic tests for our API routes. To do this, we’re going to take advantage of Django’s Test Client.

Create a new file at test/test_views.py and add the following.

import json
from django.test import TestCase, Client
from django.test import Client
from todo import models

class ToDoListTestCase(TestCase):
    """Tests for the todo list models"""

    def setUp(self):
        """Initialize the TestCase with a test client"""
        self.client = Client()
        self.test_list = models.ToDoList.objects.create(title='Test ToDo List')
        super(ToDoListTestCase, self).setUp()

    def test_todo_list_item_post_delete(self):
        """
        Test the /todo_list//items POST method to add items
        and the /todo_list/items/ DELETE method to
        remove todo list items.
        """
        # define the data to post
        post_data = {
            'title': 'Test List Item',
            'description': 'This is a test.'
        }

        # post a new ToDoListItem
        response = self.client.post(
            '/todo_lists/{0}/items'.format(self.test_list.id),
            json.dumps(post_data),
            content_type='application/json')

        # always check the response status code
        self.assertEqual(response.status_code, 200)

        # get the id of the saved object we posted
        list_item_id = response.data['id']

        # now try and delete it
        response = self.client.delete('/todo_lists/items/{0}'.format(list_item_id))

        # verify the response we got is a 204 no content
        self.assertEqual(response.status_code, 204)

    def test_todo_list_get_post_delete(self):
        """
        Test the /todo_lists GET POST and DELETE methods.
        """
        # define data in a dictionary to post to the API
        post_data = {
            'title': 'Test List',
            'items': [
                {
                    'title': 'Test Item 1',
                    'description': 'This is test item 1'
                },
                {
                    'title': 'Test Item 2',
                    'description': 'This is test item 2'
                }
            ]
        }

        # try to post the data as json
        response = self.client.post('/todo_lists', 
            json.dumps(post_data),
            content_type="application/json")

        # always assert that your status code is what you expect
        self.assertEqual(response.status_code, 200)

        # get the todo list id from the response
        todo_list_id = response.data['id']

        # now retrieve all the todo lists
        response = self.client.get('/todo_lists')
        self.assertEqual(response.status_code, 200)
        # get the todo lists from the endpoint
        received_todos = response.data

        # search for the todo list we posted
        is_present = False
        found_list = None
        for received_todo_list in received_todos:
            if received_todo_list['id'] == todo_list_id:
                is_present = True
                found_list = received_todo_list
                break

        # assert that the list we posted was indeed found
        self.assertTrue(is_present)
        # verify the list title is correct
        self.assertEqual(found_list['title'], post_data['title'])
        # verify the length of the items list is correct
        self.assertEqual(len(found_list['items']), len(post_data['items']))

        # for each ToDoListItem verify it's data is correct
        for i in range(0, len(found_list['items'])):
            found = found_list['items'][i]
            expected = post_data['items'][i]
            self.assertEqual(found['title'], expected['title'])
            self.assertEqual(found['description'], expected['description'])
        
        # delete the todo list now
        response = self.client.delete('/todo_lists/{0}'.format(todo_list_id)) 
        # verify we get a 204 HTTP No Content Response
        self.assertEqual(response.status_code, 204)

And that’s it! Run the tests again, and watch the glorious output.

Using Postman

But what if I just want to poke around at my API a little manually? To manually send requests to my APIs, I use a tool called Postman. This lets you create and save requests within this application and execute them against your API. However, I don’t want to make this tutorial about using Postman, as it’s an unnecessary step and just something I thought I’d mention. Instead, check out their awesome documentation.

Conclusion

Awesome! We now have a functioning Django API for our ToDo project. If you want to keep going, there’s some functionality that you could still add.

As an exercise, try doing the following:

  • An endpoint for getting single ToDoListItems.
  • Endpoints for updating existing ToDoLists and ToDoListItems
  • Additional testing in test_models.pytest_serializers.py, and test_views.py.
  • Refactor the code to have a separate file per model/view/serialzer. E.g. create a views/ folder with a file for each view class within it. It’s the best practice to split Python files before they get too large. Splitting up these files by classes just makes the most sense.

I hope this helped you to get started creating RESTful APIs with Django.

If you would like to see the entire project setup, the GitHub repository is here.

As a reminder, this tutorial is part of a series on Building a Web App From the Ground Up. So far we’ve done our due diligence with designing our application, we’ve initialized our API project, and now we’ve developed it as well.

The next part in this series is going to cover building a lightweight client-side app with ReactJS. This will be available by Tuesday, September 2017.  In order to focus on the quality and depth of my posts, I’ve decided to start publishing on a bi-weekly schedule (every two weeks) . This will allow me to go further in depth with my posts and provide more detailed information. The next post in this series will be available on 9/12.

Stuck? Have additional questions? Want to offer up some of your own input? Leave a comment below and I’ll get back to you.

Showing 6 comments
  • William
    Reply

    Great. Thank you.

  • Bob
    Reply

    Hey Michael, thanks for this nice tutorial, really useful.

    2 fixes:

    1)
    views.py line 93 HTTP_400_BAD_REQUEST needs a status. in front of it

    2)
    urls.py

    python manage.py test
    django.core.exceptions.ImproperlyConfigured: “^todo_lists/(?P[0-9]+)$” is not a valid regular expression: unknown extension ?P[ at position 13

    • Bob
      Reply

      This passes the tests:

      url(r’^todo_lists/(?P[0-9]+)$’, views.ToDoList.as_view()),
      url(r’^todo_lists/(?P[0-9]+)/items$’, views.ToDoListItem.as_view()),
      url(r’^todo_lists/items/(?P[0-9]+)$’, views.ToDoListItem.as_view()),

  • SEO Web Dev
    Reply

    Hi Michael, I’m stuck on this tutorial I think there is a conflict with the models.py which is found here: src/todo_api/todo/test_models.py but it is located on src/todo/test_models.py meaning it is not inside test_api folder I’ve also compare with your files on git repositories it’s not inside todo_api but outside of it.

    • Michael Washburn Jr.
      Reply

      You’re absolutely correct. I’ll update that name change immediately in my post. I did some late-stage restructuring to try and simplify the project structure a little more and forgot to fix this. Everything else working fine for you so far?

Leave a Comment