Blog

Home / Blog

Cython - Python at the Speed of C

William Tsu
Data Analyst
Experienced data analyst working with data visualization, cloud computing and ETL solutions.
May 08, 2020


Python

A high-level, general-purpose programming language with a design philosophy that emphasizes code readability. Python intelligently uses a significant white-space. Programmers use Python for creating or writing clear, logical codes for both small as well as large-scale projects. Python has a penchant for supporting multiple programming paradigms that include structured or procedural, object-oriented as well as functional programming prototypes. Most of the operating systems come with Python interpreters. CPython is used for an open-source reference implementation. The Python Software Foundation uses its resources to manage the development of both Python as well as CPython.

Cython

Cython, released in 2007, is a combination of Python, and the speed of native code C. It is well known that although Python has a rich reputation of being convenient, useful, and richly outfitted programming language, it does not have much execution speed as is needed or sought after. Enter Cython, with the characteristics of Python as well as the execution speed of programing language C. Cython has improved the yield with much-needed performance boosts, which ranges through many magnitudes.

Cython aids in side-stepping many of Python's limitations by keeping the advantages of Python intact throughout.

Combining or Compiling Python using C

The C modules used for developing or designing Cython programming language consist of generic C libraries as well as those libraries that are built specifically to work with Python. The second kind of module is generated by Cython: this can interact & communicate with the internals of Python. It can easily be bundled into any existing Python codes.

The C modules used for developing or designing Cython programming language consist of generic C libraries as well as those libraries that are built specifically to work with Python. The second kind of module is generated by Cython: this can interact & communicate with the internals of Python. It can easily be bundled into any existing Python codes.

The above-said approach dovetails the nature of software performance issues. In almost all of the programs, it is observed that a vast majority of CPU-intensive code gets concentrated in a few hot-spots. Pareto Principle is instituted here where 80% of effects come from 20% of causes (the 80/20 rule). Hence, the critical pieces of Python or the hotspots in this programming language can get incrementally translated to Cython. A performance gain is obtained where it matters the most. The rest of the program stays in Python: convenient to software programmers & developers.

Advantages of Cython

cython

As is known, Cython is an expert in speeding up codes that are already written. There are several other advantages of Cython that should be looked upon.

Faster linking up with external C libraries

It is tedious to switch back-and-forth between Python and underlying as well as external C libraries. This slows down the software considerably. This is where, Cython comes and establishes a direct link with the said, external C libraries; makes the whole process considerably faster. Cython also links with C++ libraries; a convenience indeed.

Both C & Python memory can be managed

As is known, Python objects are memory managed as well as garbage collected: the same as that of regular Python. Cython allows the proper management of both C as well as Python memories, making the job much easier than before.

Safety or Speed: As per the needs

Cython is programmed to automatically perform runtime checks for any common problems, which might pop up in C. Some of these issues include out-of-bounds access on an array. With constant checking, Cython ensures the safety of the program. The C codes generated by Cython is safer by default as it is not hand-rolled ones. There are options to disable the runtime checks for improving the speed of the software program.

Cython allows the programmers & developers to natively access Python structures, which uses the buffer protocol: allows direct access to data stored in memory. The memory of Cython lets the programmers work with all Python structures at a high speed, with the appropriate level of safety apt for the task at hand.

Cython C codes benefit from releasing Global Interpreter Lock (GIL)

The Global Interpreter Lock (GIL) of Python helps in synchronizing various threads within the interpreter. This indeed protects the access to Python objects along with managing content for available resources. The GIL can be released at certain sections for enhancing the speed of the software programming & Cython benefits from this process as well.

Cython uses Python type-hinting syntax

The type-hinting syntax of Python is mainly used by the code checkers; checking for any inconsistencies. This is used instead of the CPython Interpreter. Cython comes with its custom syntaxes for code-decorations. However, recent revisions allow Cython to use the Python type-hinting syntax.

Cython helps in obscuring sensitive Python codes

Protecting any & all sensitive information of Python applications, when they are distributed to end users can be done using Cython. This is made possible by compiling the Python modules with Cython; a side-effect of Cython's capabilities that has become quite useful.

Cython Limitations

There are limitations to this magic software programming application as well. It is imperative to discuss these limitations as well.

A little speed-up for Python codes

It is not possible for Cython to completely translate the Python codes into C language. Cython works by transforming the Python codes into a series of C calls, inside the internals of Python. This process removes Python's interpreter out of the execution loop. The process gives a speed of about 15-20% to the codes, in the best-case scenario.

A little speed-up for native Python data structures

Python is known to provide a slew of data structures that include strings, lists, and dictionaries to name a few; which are convenient to developers & programmers. However, it is slower than pure or original C.

Although via Cython, all Python data structures can be used, it runs at a slower speed. This is because Cython obtains all C APIs (Application Program Interfaces) in Python runtime and manipulates those, accordingly. These Python data structures behave like Cython-optimized Python codes. A boost might be recorded, but maybe a little.

Cython code runs the fastest only with Pure C

If Cython is allowed to work with functions or applications tagged with Pure C, it will run fast as far the Pure C takes it. However, if Cython encounters any Python data-structures, or gets called into working with an internal Python API, a performance bottleneck might be the result.

Cython is good at finding these performance bottlenecks via source-code reports. These reports show the parts of Cython that are Pure C and those that interact with Python. An optimized Cython application will have less interaction with Python data-structures or internal APIs.

The profiling and performance of Cython

The best performance from any piece can be obtained by the code-profiling process. This allows observing the performance bottlenecks. Cython is good at providing hooks for Python's cProfile Modules; Python's profiling tools can be used in Cython as well due to this. The less there is shuttling between Cython & Python, the app runs smooth and in a streamlined fashion along with being fast. Iterating objects over Cython modules instead of Python is very helpful for managing all kinds of related data; a good module to emulate while formulating the codes. A good balance between Cython & Python gives the best of both worlds while working with these for software programming as well as development.

Cython and NumPy

NumPy is a number-crunching library and Cython markedly increases its performance when it is run along with this library. As Cython codes get compiled to C. it interacts with libraries such as NumPy and aids in taking out the performance bottlenecks that can happen in the presence of Python. Cython gets good & native support for specific constructions and offers fast access to NumPy arrays. NumPy syntaxes used in Python can be used in Cython as well. The closest possible bindings between Cython and NumPy are possible via customized syntaxes of Cython. These can allow Cython codes to observe C-level constructs in various libraries at the fastest compiling time and bonding as possible. Cython does support NumPy out-of-the-box as well and further decorations can be added on to enhance the performance.

Wrapping Up

Cython markedly improves the applications run with Python as it is very compatible & runs fast when applied with Pure C. A speed improvement of about 15-20% can be observed when Cython is used along with Python data-structures as well as NumPy syntaxes. Thus, Cython is indeed an application that works at the speed of Pure C as Cython is made with Python as well as the speed of native code C. Enjoy the benefits of Cython by incorporating it along with programs & applications that run with Python.