Python Entry Points

So I recently wrote a small post on __main__.py and got a ton of feedback. Lots of readers were saying to explore Python “entry points” and how they can also act a solution in this space. So that is exactly what I did.

Our example project: Animals

animals will be the sample project we will use to demonstrate how entry points work. The project looks like this:

.
├── animals
│   ├── args.py
│   ├── __init__.py
│   ├── __main__.py
│   └── models.py
├── README.md
├── requirements.txt
└── setup.py

args.py will handle arguments:

# args.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument(
    '-k',
    '--kind',
    help='Pass the kind of animal you want to talk',
    type=str,
    dest='kind',
    default='dog'
)
args = parser.parse_args()

models.py will hold some animal classes that have a “talk” function.

# models.py
class Cat(object):
    def talk(self):
        print('Meow!')

class Dog(object):
    def talk(self):
        print('Woof!')

setup.py defines our entry points. Entry points allow us to register Interpreter-level objects (typically any Python type that is callable, i.e. classes & functions). When these are registered, The objects are able to be retrieved from any Python module installed. Essentially, they can act as an interface through which we can inject plugins into other modules (More on this later!).

# setup.py
from setuptools import setup

setup(
    name='animals',
    entry_points={
        'console_scripts': [
            'animals=animals.__main__:main'
        ],
        'animals': [
            'cat=animals.models:Cat',
            'dog=animals.models:Dog',
        ],
    }
)

console_scripts, and animals are our entry points (or registries) in the above setup.py. console_scripts is especially unique because once we install this module, the callable is available to use in our terminal. i.e.:

$ pip install -e .
$ animals --kind dog
Woof!
$ animals --kind cat
Meow!

This is great. If all you wanted to do was to have a clean console script then entry points has satisfied your requirements. But we can do more!

__main__.py is our primary driver and holds our logic.

import pkg_resources
from .args import args

def main():
    animals = {}
    input_animal = args.kind
    for entry_point in pkg_resources.iter_entry_points('animals'):
        animals[entry_point.name] = entry_point.load()

    animal = animals[input_animal]()
    animal.talk()

if __name__ == '__main__':
    main()

Console scripts allows us to is to tap into the __main__.py‘s main function by way of the line pkg_resources.iter_entry_points(‘animals’). Any object that is registered under ‘animals’ will be here. We load them all into the program. We then use our command line arguments to select the one the user wants, and then call it, and have it talk.

animal = animals[input_animal]()
animal.talk()

The entry points cat and dog are in setup.py, so we know they are registered. Maybe there are others registered from other modules? This is sounds like a “plugin” to me.

Using Entry Points to Create Plugins

This is where entry points get insanely cool. What if we want to use the animals engine but with another animal besides Dog and Cat? It’s easy. Animals gets its classes from entry points. So it can see any object that’s registered an animals entry point, regardless of what module it belongs to. All we would need to do is register an animal and it would be available instantly. For example, what if we wanted to add the animal Cow? Here’s our cow plugin.

.
├── cow-plugin
│   ├── cow.py
│   └── setup.py

cow.py

# cow.py
class Cow(object):
    def talk(self):
        print('Mooooo!')

def main():
    Cow().talk()

setup.py

from setuptools import setup

setup(
    name='cow',
    entry_points={
        'console_scripts': [
            'cow=cow:main'
        ],
        'animals': [
            'cow=cow:Cow',
        ],
    }
)

It’s key here that we register our Cow class under the ‘animals’ entry point. Let’s see if it works.

$ pip install -e cow-plugin
$ animals --kind cow
Moo!

Woo! We are successfully leveraging the animals engine, but with our Cow class. It’s being loaded dynamically at runtime and the author of animals does not need to make any changes to the engine. It’s a plugin.

Good luck and happy hacking!

Use __main__.py

We have all seen the __init__.py file and know its role, but what is __main__.py? I have seen many Python projects either on Github or at work that do not take advantage of this magic file. In my opinion, including a __main__.py is a better way to interface with multi-file python modules.

But first some background: How do most people run their Python programs?

Occasionally you will write a program that is organized enough so that it can both be imported as a module to another program or run by itself through a command line interface. In this case, you are probably familiar with the following lines commonly placed at the bottom of the file:

if __name__ == '__main__':
    main(sys.argv)

When you run a program by calling the Python interpreter on it, the magic global variable __name__ gets set to __main__. We can use that to know that we are not being imported as a module, but ran. For example:

python myapp.py

This works just fine for single file projects.

The problem

If you are like me, you do not want your entire application in a single python file. Separating related logic out into their own files makes for easier editing and maintainability. For example:

.
├── README.me
├── requirements.txt
├── setup.py
└── src
    ├── __init__.py
    ├── client.py
    ├── logic.py
    ├── models.py
    └── run.py

But then this raises confusion to a user who just downloaded your program. Which one is the main file? Is it run.py? Or maybe client.py? What file has my precious if __name__ == ‘__main__’ statement? This is where we can get a lot of value from __main__.py.

__main__.py

The magic file __main__.py is called when you run your project with the -m module flag. If you code is intended to be used a module first, and command line interface second, this makes perfect sense to use. Think of it as a place we can put whatever would be in our body of our if __name__ == __main__ statement. Refactoring the project above to conform:

.
├── README.me
├── requirements.txt
├── setup.py
└── myapp
    ├── __init__.py
    ├── __main__.py
    ├── client.py
    ├── logic.py
    ├── models.py

And now we just run it as a module not a singleton program.

python -m myapp

__main__.py will auto-magically be executed. This is a perfect place to put a command line interface and handle program arguments!

Good luck and happy hacking!

My Python VS Code Setup

If you are like me, you have dropped your other editors and IDEs for VS Code. I have been using VS Code as my primary editor for about a year now. My goal is to balance the bare minimum with the most efficient plugins. The last thing we want to do is turn VS Code into Intellij, right? Here are my top plugins and configurations.

Python

This plugin is essential. It provides a plethora of features for working with Python files. If you have ever worked with Python in VS Code you probably already have it installed!

ImportMagic

This is a plugin whose features I think will eventually be part of VS Code core. I do not like typing imports manually as there is a lot of room for error. Let’s be honest, no one can remember in which module the class you want to instantiate is in. For those of you coming from PyCharm/Intellij, you will be missing your auto-imports until you install this one.

autoDocstring

Probably the most underrated of the Python plugins — autoDocstring makes writing doc strings a breeze, and who doesn’t like docstrings? It is configurable for all the most popular docstring formats like Google, Sphinx, and others.

SQLite

SQLite – everyone’s favorite database. It deserves a proper database viewer inside VS Code. This plugin is very lightweight and is great to have when working on your Django apps.

Vim

This one is totally preference! I like editing using vim keybinds. I feel like I have more control, and its easier to navigate files. However, if you have never used Vim before, then take some time to learn about it and try it out somewhere else before bringing it into your editor. However, I must add that I often toggle Vim off in VS Code when I want, and then turn it back on when I am ready to start coding.

Gitlens

Not unique for Python development, but all of my Python code is checked into Git making this a no-brainer choice. Some people like to turn off the Blame features because they feel as if it clutters the file. I personally do not mind them, but to each their own.

Atom One Dark Theme

The one thing that Atom is unquestionably better than VS Code at is it’s dark theme. This theme is so nice on the eyes. Personally, I think it’s the best color scheme and I hope that all of the software that I use incorporates these dark pastel shades in one form or another.

Django

A must-have for Django developers out there. It is especially useful if you do any server side rendering using Django templates, or even Email templates. It will provide syntax highlighting and snippets. Add a rule to your settings, or Ctrl + K M and select Django-html.

Settings

I like to keep my settings pretty lean, but there are also a few essential flags to set to make your life much easier! Here are some I like:

{
  "editor.formatOnSave": true,
  "editor.minimap.enabled": false,
  "files.autoSave": "afterDelay",
  "files.autoSaveDelay": 500,
  "autoDocstring.docstringFormat": "google",
  "editor.snippetSuggestions": "top"
  "files.watcherExclude": {
    "**/.git/objects/**": true,
    "**/.git/subtree-cache/**": true,
    "**/node_modules/**": true,
    "**/.direnv/**": true,
    "**/.venv/**": true
  }
}

Thanks for reading! And happy hacking!

Effective Python API testing using Django and Patch

Today I would like to offer a brief showcase on how to effectively test a Django API. Essentially, what we want to accomplish is the ability to test and control how our applications handle specific responses from external web APIs. A program might make calls to several different APIs like Reddit’s or Google’s, and we want to capture these responses under test. More importantly, in a way that our tests can pass every time regardless of the status of the APIs we consume.

Continue reading “Effective Python API testing using Django and Patch”

New look!

I redesigned the site as you could probably tell. I moved off of my own homebrewed creation to WordPress because its more in line with my vision, and frankly I don’t have time to develop new features. Happy hacking!