[NMLUG] Python tight loop causing massive CPU barfage

[paul.tietjens at moriarty.k12.nm.us (Paul Tietjens)]

Peter Espen wrote:

>
> I used the following python code to open 175,000 mail files, read the 
> first 1024 lines from each file and search for a line beginning with 
> "Subject" from each.
>
> It took 2 minutes 24 seconds on a 800MHz Pentium III running SuSE 9.0:
>
> real    2m24.097s
> user    2m10.270s
> sys     0m9.530s
>
>
> Here's the read/search loop for each file:
>
>     myre = re.compile('^Subject')
>
>     for fn in filelist:
>
>        mode = os.stat(fn)[stat.ST_MODE]
>
>        if stat.S_ISREG(mode):
>
>            mf_fd = os.open( fn, os.O_RDONLY )
>            the_file = os.fdopen(mf_fd, "r", 1024)
>            the_lines = the_file.readlines(1024)
>            for line in the_lines:
>                m = myre.match(line)
>            the_file.close()
>

Wow!  Do you have the rest of that?  I'm not sure what module I need for 
'stat' to work, but after import 'os' I stil l get:

Traceback (most recent call last):
  File "./foo.py", line 10, in ?
    mode = os.stat(fn)[stat.ST_MODE]
NameError: name 'stat' is not defined