Python with multiprocessing and timeout

Nowadays, we have a lot of cores in the CPU. But we cannot make any good use of it unless we program in multicore.

Say we have thousands of SEC filings to process every year; we write a function to collect data from it. For most filings, it works just fine. But for a small group of filings (8 out of 8000+ in 2005, for my data set, as an example), it runs with no end. 10 hours later, the process for this filing is still stuck somewhere. As Obama once famously (and maybe unfairly) said, “Eight is enough.” (It was unfair, according to McCain, because Obama made Bush a pet name for him.) We can put a timeout exception to end it.

Let’s first make a troublesome function, one that sometimes runs overtime.

from time import sleep

def f(x):
    """ x can be any integer. 
        But when x is a even number, it takes x seconds to finish; 
        when x is an odd number, it takes one second. 
        When x=2, it causes an error/exception.
    sl = x if (-1)**x >0 else 1
    print("Start running with x={} and sleep={}".format(x,sl))
        print("  finished with x={} and sleep={}, result={}".format(x, sl, 1 / (x-2)))
        return x
    except Exception as e:
        print('\n\nCaught exception {} in worker thread (x = {:d}):'.format(e, x))
        return None

And, let’s make use of a 2-core CPU by using Python’s multiprocessing. Run f(x) parallelly, and collect results as a list. And every child-process is allowed to run for at most 5 seconds. In the end, we collect the result and do some analysis (that’s not the job of this post, though).

import multiprocessing
if __name__ == '__main__':
    with multiprocessing.Pool(2) as pool:
        async_results = [pool.apply_async(f, (i,)) for i in range(20)]
        for async_res in async_results:
                this_res = async_res.get(timeout=5)
            except Exception as e:
                print("Exception: {}".format(e))
        #Removing unsuccessful ones
        results_collection = [r for r in results_collection if r !=None]

The outputs:

[0, 1, None, 3, 4, 5, 6, 7, None, 9, None, 11, None, 13, None, 15, None, 17, None, 19]
[0, 1, 3, 4, 5, 6, 7, 9, 11, 13, 15, 17, 19]

The first line of output has 20 elements. The return value of f(x) for x in range(20). Some are None (x=2, 8, 10, 12, 14, 16, 18) because it either had an error or was timed out.

And the final result is ready for analysis next step: [0, 1, 3, 4, 5, 6, 7, 9, 11, 13, 15, 17, 19]

How it runs live:

The code in GitHub:

The code looks too simple. Why would I bother writing a post in my blog? Well, to me, yesterday it was not this simple. I tried a few other ways. They didn’t work. The worst case was, my workstation was even frozen, and I cannot run any command even in bash. There was an error message from the OS: “-bash: fork: retry: Resource temporarily unavailable.” And I googled a way out… “exec killall python3”

What can we learn about tax from the financial statement?

Dear Professor Dr. ▮▮▮▮▮▮▮▮▮▮▮,

I’m a student from your class TAX 101 and I’m preparing for the second exam. I wish to ask a few questions about the case Apple’s Tax Expenses.
From the income statement of Apple’s annual report 2017, we know that Apple has Income before provision for income taxes 64,089 and provision for income taxes 15,738, from which we get the tax rate of 24.6%. Is this the “Book ETR” you mentioned in the lecture?And, from the notes we see Apple defer 5,966 of the 15,738 provision for income taxes, leaving the current component only 9,772. Is this the actual tax payable? And is the corresponding tax rate 15.2% the so called “Cash ETR”? 

I was trying to figure out what information we can extract out of this reconciliation table. Also, I come up with a paper Henlon (2003) who criticized a lot about this calculation and implied this is wrong and that is also wrong… She was our ZEW guest keynote speaker… I’m so confused and worried about the exam. Could you please help me, professor?

Hanlon, Michelle. “What can we infer about a firm’s taxable income from its financial statements?.” National Tax Journal(2003): 831-863.

Income before provision for income taxes64,089  
Computed expected tax22,43135% 
State tax1850% 
Indefinitely invested earnings of foreign subsidiaries-6,135-10% 
Domestic production activities deduction-2090% 
Research and Development credit-678-1% 
Provision for income taxes15,73824.6%(1)
of which current9,77215.2%(2)
of which deferred5,9669.3% 

Best wishes,


Reeyarn Zhiyang Li

University of Mannheim | Business School
Schloss | 68131 Mannheim | Germany | Phone +49 (0) 621 ▮▮▮ ▮▮▮▮
Email: ▮▮▮▮

Dear Reeyarn,

Look at the Statement of Cash Flows and you find a line which reads

Cash paid for income taxes, net: 11,591,

…from which you get the cash ETR of 18.1%.

Best regards,


Hello world!

Welcome to

Yes. This is a classic Hello world! page generated by WordPress. And I am a big fan of Hello world! Anyone who took a course in any programming language must have learned writing a Hello World! code.

Yes. I think of myself as an amateur programmer. And I am also trying to become a data scientist. The world may be heading toward a future where human beings are no longer needed, and Artificial Intelligence could one day become the dominating intelligent beings. We are in a serious competition, fighting for the survival of human beings. Learn maths, learn programming, learn machine learning, and learn to survive.

Reeyarn Zhiyang Li