2025-11-22 –, Breakout Room
This talk will explore the various ways of spawning processes in Python (subprocess, multiprocessing, concurrent.futures, fork, vfork, posix_spawn, etc.) and how those implementations inside Python have been leading to regular dead-locks and hard crashes for many years in production environments.
This talk will introduce the multiple different ways of spawning sub-processes in Python with simple & easy to understand examples. We will then have a brief look at the different implementations and their various use-cases.
Then we will show how spawning processes in Python has always been fraught with problems for over 15 years. We will show real-world examples of seemingly unexplainable dead-locks and hard crashes when spawning processes in Python, especially in the context of multi-threaded applications.
Lastly, we explain the reasons for those hard-to-debug dead-locks & hard crashes, show the problems that are associated to this day with using os.fork() & os.register_at_fork(). Then we will offer up how alternatives like os.posix_spawn() + vfork can help to make your Python programs more robust.
Intermediate
Teijo Holzer has been working in the Visual Effects industry for WetaFX in Wellington, New Zealand for over 15 years on feature films like Avatar & The Hobbit. He has over 15 years of professional Python experience, and has been instrumental in discovering extremely difficult bugs in Python & associated libraries (e.g. PySide), providing reproducible test cases and suggestions for fixes.