Simplify Python with fastcoreSimplify Python with fastcore

Why pay attention to fastcore?

[fastcore](https://github.com/AnswerDotAI/fastcore) is described as *"Python goodies to make your coding faster, easier, and more maintainable"*. It is maintained by [Answer.ai](https://www.answer.ai) and used in many of its projects. In my opinion, `fastcore` is highly underrated, especially outside of the [Answer.ai](https://www.answer.ai)/[fast.ai](https://www.fast.ai) community. I will show you 10 hidden gems that will make your Python code more readable and maintainable. These features also allow you to iterate quicker in your Python development. Many `fastcore` functions are one-liners that makes programming Python smooth and more fun. To highlight the benefits, I will compare writing code without fastcore and with fastcore.

0. Import

The easiest way to work with fastcore is to treat it as an extension of the Python standard library and import all functions:
from fastcore.all import *

1. parallel

Parallelizing functions is something we often encounter. Writing this code can be annoying and often makes code ugly. `fastcore` has a [parallel](https://fastcore.fast.ai/parallel.html#parallel) function that just works out of the box. Let's say we want to square all numbers from 0 to 5.
def square(x): return x**2
list_items = list(range(5))
Without fastcore, you would have to write something like this:
from concurrent.futures import ProcessPoolExecutor
                     
with ProcessPoolExecutor() as executor:
    results = list(executor.map(square, list_items))
results # [0, 1, 4, 9, 16]
With fastcore:
parallel(square, list_items) # [0, 1, 4, 9, 16]
This is a lot easier to read and write. `parallel` can also be customized. For example, set `n_workers` to specify the number of workers, `progress=True` to show a progress bar and `threadpool=True` to use a threadpool instead of a process pool. If you want to parallelize an async function, use [parallel_async](https://fastcore.fast.ai/parallel.html#parallel_async).

2. L

[L](https://fastcore.fast.ai/tour.html#l) is a fascinating custom implementation of `list`. `L` augments list functionality with ideas from functional programming languages, like [Perl](https://www.perl.org) and [Haskell](https://www.haskell.org). For example, take the case of creating `range(10)`, square all numbers, filter down to numbers larger than 10 and shuffle them. This is a tough task to write as a one-liner in pure Python so we would resort to something like this:
from random import shuffle
squares = [i**2 for i in range(10)]
squares_above_10 = [i for i in squares if i > 10]
shuffle(squares_above_10)
squares_above_10 # [16, 36, 49, 64, 81]
With `L`, we can construct a readable one-liner:
L.range(10).map(lambda x: x**2).filter(lambda x: x > 10).shuffle() # [16, 36, 49, 64, 81]
`L` has a lot more interesting functionality like this, which can be explored on [here](https://fastcore.fast.ai/foundation.html#l). You could definitely write an entire blog post on `L` alone.

3. patch

[patch](https://fastcore.fast.ai/basics.html#patch) is a convenient way to add methods on classes outside of the class definition (i.e. [monkey patching](https://en.wikipedia.org/wiki/Monkey_patch)). Use cases include iteratively working on a class or adding functionality to classes from an installed library. This allows you to leverage more of Python's dynamic nature, which can be very powerful. It is heavily used in [Answer.ai projects](https://github.com/AnswerDotAI) and allows them to iterate quickly, especially combined with building [libraries in Jupyter notebooks](https://github.com/AnswerDotAI/nbdev).
Let's take the hypothetical case of building a Python program for a drivers license exam centre. We have a dataclass called `Person` to represent an exam candidate. It should hold `name`, `age` and if the person passed a theory exam (`passed_theory_exam`). We would like to clearly show the person's name and age with a `__repr__`. If the person is over 18 and passed the theory exam they are allowed to take a driving test, which we need a method for. Without fastcore you would probably write the dataclass like this:
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int
    passed_theory_exam: bool = False
    
    def can_take_driving_test(self):
        return self.age >= 18 and self.passed_theory_exam
                     
    def __repr__(self):
        return f"{self.name} is {self.age} years old"
                     
p = Person("Alice", 25, True)
print(p) # Alice is 25 years old
print(p.can_take_driving_test()) # True
Monkey patching without fastcore can be done, but generally requires weird and clunky code:
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int
    passed_theory_exam: bool = False

def can_take_driving_test(self):
    return self.age >= 18 and self.passed_theory_exam
                     
def person_repr(self):
    return f"{self.name} is {self.age} years old"

Person.can_take_driving_test = can_take_driving_test
Person.__repr__ = person_repr
p = Person("Alice", 25, True)
print(p) # Alice is 25 years old
print(p.can_take_driving_test()) # True
With fastcore, you can gradually patch methods on the class. This is especially powerful for [literate programming](https://en.wikipedia.org/wiki/Literate_programming) where you iteratively add, test and explain functionality in the same document. A framework like [nbdev](https://nbdev.fast.ai) lets you do literate programming with Jupyter notebooks.
from dataclasses import dataclass

@dataclass
class Person:
    name: str
    age: int
    passed_theory_exam: bool = False
                     
p = Person("Alice", 25, True)
                    
@patch
def can_take_driving_test(self:Person):
    return self.age >= 18 and self.passed_theory_exam
                     
print(p.can_take_driving_test()) # True
                     
@patch
def __repr__(self:Person):
    return f"{self.name} is {self.age} years old"
                     
print(p) # Alice is 25 years old
The `patch` decorator looks at the type annotation of `self` and patches the class with that method. There are a few additional options for patching explained [here](https://fastcore.fast.ai/basics.html#patching).

4. save/load pickle

`save_pickle` and `load_pickle` work as advertised. Single functions for pickling so you don't have to remember and write the boilerplate `with` statements. Without fastcore:
import pickle

with open('my_data.pkl', 'wb') as f:
    pickle.dump(my_data, f)

with open('my_data.pkl', 'rb') as f:
    data = pickle.load(f)
With fastcore:
save_pickle(my_data, 'my_data.pkl')
data = load_pickle('my_data.pkl')
A small downside is that, at the time of writing, `save_pickle` and `load_pickle` don't support custom arguments for `pickle`. Saving pickle objects can often be optimized by setting `protocol=5`.

5. globtastic

[globtastic](https://fastcore.fast.ai/xtras.html#globtastic) is an extension of the [glob](https://docs.python.org/3/library/glob.html) module for parsing files and directories. `globtastic` augments `glob` with regex matching, detailed skipping options and symlink handling. Let's say you want to retrieve all Python files with the word `test` in it, but exclude hidden folders and the `__pycache__` folder. This is a realistic operation a library like [pytest](https://docs.pytest.org/en/stable/) has to perform.
To the best of my knowledge this becomes a tricky list comprehension without fastcore, even if we use [pathlib](https://docs.python.org/3/library/pathlib.html): 
import glob
from pathlib import Path

[f'./{f}' for f in glob.glob('**/*test*.py', recursive=True) if not any(part.startswith(('.', '__')) for part in Path(f).parts)]
# ['./test_file1.py', './test_file2.py', './file_with_tests.py']
Hopefully you are starting to see a pattern around `fastcore`. With `fastcore` this collapses into a beautiful one-liner.
globtastic('.', file_glob='*test*.py', skip_folder_re='^[_.]|__pycache__')
# ['./test_file1.py', './test_file2.py', './file_with_tests.py']

6. dict2obj

[dict2obj](https://fastcore.fast.ai/xtras.html#dict2obj) allows you to access values as attributes (i.e. dot notation) for dictionaries. Without fastcore, you would have to write this object yourself using `__getattr__` and `__setattr__`:
class AttrDict(dict):
    def __getattr__(self, key): return self[key]
    def __setattr__(self, key, value): self[key] = value
                     
d = AttrDict({'a': 1, 'b': 2})
d.a # 1
d.b # 2
d.c = 3
d.c # 3
With fastcore:
d = dict2obj({'a': 1, 'b': 2})
d.a # 1
d.b # 2
d.c = 3
d.c # 3

7. listify

[listify](https://fastcore.fast.ai/basics.html#listify) turns a value into a list. If the element is already a list, it returns the list unchanged. This may sound trivial at first, but there are many cases where you want to support different inputs, but your function only accepts lists. Without fastcore, you would do something like this:
def convert_to_list(x):
    if not x:
        return []
    elif not isinstance(x, list):
        return [x]                
    return x

convert_to_list(None) # []
convert_to_list('a') # ['a']
convert_to_list(42) # [42]
convert_to_list([1, 2, 3]) # [1, 2, 3]
We can see this function is not really robust, so to write something satisfactory we would have to go down a rabbit hole of edge cases, write tests, etc. With fastcore we can replace all these cases with `listify`:
listify(None) # []
listify('a') # ['a']
listify(42) # [42]
listify([1, 2, 3]) # [1, 2, 3]

8. str2x

Many of you might be familiar with the problem of parsing strings to other types. For example, when building APIs you are sometimes dealing with parsing JSON as strings. `fastcore` provides ["str2x"](https://fastcore.fast.ai/basics.html#str2bool) functions for parsing strings. We will discuss [str2bool](https://fastcore.fast.ai/basics.html#str2bool), [str2int](https://fastcore.fast.ai/basics.html#str2int), [str2float](https://fastcore.fast.ai/basics.html#str2float) and [str2list](https://fastcore.fast.ai/basics.html#str2list). I will skip the discussion of how we would do this in pure Python as there are many edge cases to converting strings to other types. If you would like to see the implementation, check out the [fastcore str2x source code](https://github.com/AnswerDotAI/fastcore/blob/c0608379fe60014534c8dffe2e381138e8160f53/fastcore/basics.py#L1149). These utilities perform as advertised and are robust. You just need to make sure the input is a string:
# String 2 Boolean
str2bool('1') # True
str2bool('yes') # True
str2bool('y') # True
str2bool('on') # True
str2bool('off') # False
str2bool('') # False
str2bool('[1]') # TypeError: s==[1] not bool
                     
# String 2 Int
str2int('42') # 42
str2int('on') # 1
str2int('None') # 0
str2int('') # 0
                     
# String 2 Float
str2float('42.0') # 42.0
str2float('42.5') # 42.5
str2float('0') # 0.0
str2float('') # 0.0
         
# String 2 List
str2list("[None]") # [None]
str2list("[]") # []
str2list("['a',2,'42']") # ['a', 2, '42']
str2list("['a', 2, '42']") # ['a', 2, '42']

9. flatten

[flatten](https://fastcore.fast.ai/basics.html#flatten) is a really cool utility for creating a generator from a collection of flattened objects. Without fastcore I would probably use [np.ravel](https://numpy.org/doc/stable/reference/generated/numpy.ravel.html), but this only works for NumPy arrays. What if we have a weird collection like this?
weird_collection = [1, [2, 3], (4, 5), set([6, 7])]
In pure Python we could parse it like this:
def diy_flatten(x):
    for i in x:
        if isinstance(i, (int, float, complex, str, bool)): yield i
        if isinstance(i, (list, tuple, set)):
            for j in i: yield j
            
for i in diy_flatten(weird_collection):
    print(i) # 1 2 3 4 5 6 7
With fastcore we once again simplify:
for i in flatten(weird_collection):
    print(i) # 1 2 3 4 5 6 7

10. range_of

`range_of` simplifies iterating through the indices of an iterable. Without fastcore, you would have to write this:
for i in list(range(len(['a', 'b', 'c', 'd']))):
    print(i) # 0 1 2 3
With fastcore:
for i in range_of(['a', 'b', 'c', 'd']):
    print(i) # 0 1 2 3
The simplification here looks trivial, but all these little things add up and improve the readability of your code. This allows for faster iteration in your projects.

11. BONUS: Transform/Pipeline

`Transform` and `Pipeline` are powerful tools in `fastcore` for data transformations. I already wrote an [in-depth blog post](/posts/fastcore-quantum) about this, so check that out if you want to learn more.

Closing

And that is were I'll leave it! I could easily add another 10 `fastcore` functions that are awesome, but hopefully I've convinced you of its power sufficiently to start your own exploration. You can continue exploring fastcore by checking out [the docs](https://fastcore.fast.ai).
Back to Posts Overview
© Carlo Lepelaars 2024 - 𝕏