{"id":329,"date":"2020-12-24T16:45:54","date_gmt":"2020-12-25T00:45:54","guid":{"rendered":"http:\/\/nramkumar.org\/tech\/?p=329"},"modified":"2020-12-26T16:07:00","modified_gmt":"2020-12-27T00:07:00","slug":"cython-tips-and-tricks","status":"publish","type":"post","link":"https:\/\/nramkumar.org\/tech\/blog\/2020\/12\/24\/cython-tips-and-tricks\/","title":{"rendered":"Cython &#8211; Tips and Tricks"},"content":{"rendered":"\n<ul class=\"wp-block-list\"><li>Don&#8217;t overthink it &#8211; your .pyx is converted to a .c file and then converted into a Python callable extension module. This can be confusing at first &#8211; including, what language are you coding in? Essentially you are coding in Python, with some extensions\/limitations that allow you to mix in some C like constructs and code. Eventually, all of the Cython code is converted to C\/C++<\/li><li><code><a href=\"https:\/\/cython.readthedocs.io\/en\/latest\/src\/tutorial\/cdef_classes.html\">cdef<\/a><\/code> &#8211; learn it, love it, use it. This keyword is used in multiple contexts &#8211; it is used to declare variables of primitive C types with no equivalent Python types or to avoid creating\/using Python wrappers. It is also used to provide type hints for variables and parameters which allows optimizations and speeds up the code. When you use it to define a Python class, it adds some restrictions but creates a version of the class <a href=\"https:\/\/notes-on-cython.readthedocs.io\/en\/latest\/classes.html\">that is more memory efficient and faster<\/a>. When you use it for methods (class methods or free methods), it defines native only functions that have lower overhead than a Python function.<\/li><li><code>cdef<\/code> means accessible from within the Cython module and in native code only. It will not be accessible from Python code.<\/li><li><code>cpdef<\/code> can be used to generate definitions accessible from both Native and Python code.<\/li><li>Use <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\"><a href=\"https:\/\/cython.readthedocs.io\/en\/latest\/src\/reference\/extension_types.html?highlight=__cinit__#initialization-cinit-and-init\">__cinit__<\/a><\/code> and <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\">__dealloc__<\/code> for native resource management &#8211; native libraries will often return opaque pointers\/handles that are managed using custom alloc\/free functions. A great way to handle this is without leaks, taking care of allocation failures is to create a <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\">cdef<\/code> wrapper class with the resource type as a <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\">cdef<\/code> member variable and handle allocation + allocation failure handling in the special method <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\">__cinit__<\/code> and free in the <code style=\"background-color: rgb(255, 255, 255); font-size: 16px;\">__dealloc__<\/code> function.<\/li><li>You can use simple array slicing with length parameter to convert a raw, native chunk of memory into a Python byte array &#8211; for example, if you have an <code>cdef unsigned char* buf <\/code>and <code>cdef size_t sz<\/code> which is a native byte buffer with size and you need to pass it to a Python function <code>foo<\/code> that takes a byte array, simply do <code>foo(buf[:sz])<\/code> and it will work.<\/li><li>Avoid buffer copying using <a href=\"http:\/\/docs.cython.org\/en\/latest\/src\/userguide\/memoryviews.html\">typed memory views<\/a> &#8211; the approach above will still result in a copy of the native buffer into a Python bytes object during the call. If you know that you do not need the copy, you can use a typed memory view by doing <code>cdef unsigned char[:] mem_view = &lt;unsigned char [:sz]>buf<\/code> and then doing <code>foo(mem_view)<\/code><\/li><li>When wrapping a C library, <a href=\"https:\/\/cython.readthedocs.io\/en\/latest\/src\/userguide\/external_C_code.html\">you only need to declare what you need including struct members<\/a>. This can allow for concise and simple pxd files and accelerate development time.<\/li><li>You cannot use the <code>*<\/code> operator for pointer dereferencing &#8211; use array syntax instead. So instead of dereferencing <code>unsigned int* sz<\/code> with <code>*sz = 0<\/code> you should do <code>sz[0] = 0<\/code><\/li><li>Some good links on Cython<ul><li><a href=\"http:\/\/okigiveup.net\/an-introduction-to-cython\/\">An Introduction to Cython, the Secret Python Extension with Superpowers (okigiveup.net)<\/a><\/li><li><a href=\"https:\/\/phonchi.gitbooks.io\/cython_note\/content\/\">Introduction \u00b7 Cython_Note (gitbooks.io)<\/a><\/li><li><a href=\"https:\/\/cython.readthedocs.io\/en\/latest\/src\/tutorial\/\">Tutorials \u2014 Cython 3.0a6 documentation<\/a><\/li><\/ul><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Don&#8217;t overthink it &#8211; your .pyx is converted to a .c file and then converted into a Python callable extension module. This can be confusing at first &#8211; including, what language are you coding in? Essentially you are coding in Python, with some extensions\/limitations that allow you to mix in some C like constructs and&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[96],"tags":[97,98],"class_list":["post-329","post","type-post","status-publish","format-standard","hentry","category-python","tag-development","tag-programming"],"_links":{"self":[{"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/posts\/329","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/comments?post=329"}],"version-history":[{"count":4,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/posts\/329\/revisions"}],"predecessor-version":[{"id":335,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/posts\/329\/revisions\/335"}],"wp:attachment":[{"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/media?parent=329"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/categories?post=329"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nramkumar.org\/tech\/wp-json\/wp\/v2\/tags?post=329"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}