Skip to content

ThreadPoolExecutor marks interpreter as shutdown before non daemon threads exit #148280

@aaix

Description

@aaix

Bug report

Bug description:

When the main thread exits, concurrent.futures.ThreadPoolExecutor mistakenly marks the interpreter as shutdown, regardless of if there are non daemon threads still executing, this causes any work submitted to the executor to fail with RuntimeError: cannot schedule new futures after interpreter shutdown.

PoC:

import time
import threading

from concurrent.futures import ThreadPoolExecutor

def work():
    print("Starting work")
    time.sleep(1)
    print("Work complete")

def worker_thread():
    # sleep a little to give the main thread time to exit
    time.sleep(1)

    executor = ThreadPoolExecutor()
    
    future = executor.submit(work)
    

def main():
    thread = threading.Thread(target=worker_thread)
    thread.start()

    # main thread exits now

if __name__ == "__main__":
    main()

Result

Exception in thread Thread-1 (worker_thread):
Traceback (most recent call last):
  File "/usr/lib/python3.14/threading.py", line 1081, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/lib/python3.14/threading.py", line 1023, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/*****/*****/implementation/./bug.py", line 18, in worker_thread
    future = executor.submit(work)
  File "/usr/lib/python3.14/concurrent/futures/thread.py", line 207, in submit
    raise RuntimeError('cannot schedule new futures after '
                       'interpreter shutdown')
RuntimeError: cannot schedule new futures after interpreter shutdown

Interpreter is marked as shutdown here:

def _python_exit():
global _shutdown
with _global_shutdown_lock:
_shutdown = True
items = list(_threads_queues.items())
for t, q in items:
q.put(None)
for t, q in items:
t.join()
# Register for `_python_exit()` to be called just before joining all
# non-daemon threads. This is used instead of `atexit.register()` for
# compatibility with subinterpreters, which no longer support daemon threads.
# See bpo-39812 for context.
threading._register_atexit(_python_exit)

Expected Result:
All non daemon threads should be able to use ThreadPoolExecutor until they exit.

example was tested on 3.14.0

cc @ZeroIntensity

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Metadata

Metadata

Assignees

Labels

3.15new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytriagedThe issue has been accepted as valid by a triager.type-bugAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions