Project does not appear to work with version Python 3.11.4

[MikeKav]

Is there any guide to using this with Python 3.11.4, or has anyone had any success adapting this repository?

I wasn't sure whether the modules in /parsers were older system python modules, or specific versions needed to be in the sub directory for the project to work. The syntax in some of the files in the /parsers directory seemed to indicate they only supported python2

In my initial test, the initial problem was that when try to run snapshot.py there was a problem with the line in /parsers/parser.py 'import parser'

From the web, Python built-in parser module is deprecated in Python 3.9 and will be removed in Python 3.10.

I tried running the project in Python 3.9.0.

(venv3.9) C:\temp\procfs-snapshot>python snapshot.py Traceback (most recent call last): File "C:\temp\procfs-snapshot\snapshot.py", line 9, in from parsers.tail import read_tailed_files File "C:\temp\procfs-snapshot\parsers__init__.py", line 2, in import loadavg ModuleNotFoundError: No module named 'loadavg'

(venv3.9) C:\temp\procfs-snapshot>pip install loadavg Requirement already satisfied: loadavg in c:\temp\procfs-snapshot\venv3.9\lib\site-packages (1.9.0)

[darrengarvey]

Hi @MikeKav. It's possible there needs to be a few fixes to work with python 3, but they should be minimal - most of the repo is intentionally simple python.

As for the issues you see here, looks like you'll need to set PYTHONPATH environment var to the root of the repo. loadavg and parser are contained in the repo.

Happy to give more help on this.

[MikeKav]

Thanks @darrengarvey, I've broken my python setup trying to run earlier versions so I'll try uninstalling and reinstalling python 3.11 again and try the env var you suggest. I wasn't sure if those files contained in the repo should be used as they look to have python 2 syntax, for the print statements at least: /parsers/parser.py

class Parser(object):
    def parse(self, *args, **kwargs):
        print 'Dont know how to parse you yuou!!'

[MikeKav]

Python 3.11 installed:

Changes required:

set PYTHONPATH=c:\temp\procfs-snapshot\parsers

Code Changes File "C:\temp\procfs-snapshot\parsers\parser.py", line 4 SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?

FIX: print('Dont know how to parse you!')

File "C:\temp\procfs-snapshot\model.py", line 25 self.start_addr = 0L ^ SyntaxError: invalid decimal literal

FIX: Lines 25 - 28 self.start_addr = 0 self.end_addr = 0 self.offset = 0

File "C:\temp\procfs-snapshot\util.py", line 15, in find_all_subclasses if not subclasses.has_key(name): ^^^^^^^^^^^^^^^^^^ AttributeError: 'dict' object has no attribute 'has_key'

FIX: if name not in subclasses:

parsers\smaps.py replace all long references with int

=========

This has got some of the code running, it looks like connection works as the -v options shows various process information I am expecting from the remote host, however the below command fails

python snapshot.py --host 192.168.0.78 --password xxxx -d data.sql -c 1 --period 1 2023-06-09 16:12:47,261 INFO snapshot.py:114 Reading procfs with cmd: bash -c "nice tail -v -n +1 /proc//{cmdline,smaps} /proc/meminfo /proc/loadavg /proc/uptime /proc/vmstat 2>/dev/null; nice find /proc/ -type f -name stat -exec tail -v -n +1 {} \; 2>/dev/null | awk '/==>/ {print} /^[0-9]/ {print \$2, \$10, \$12, \$14, \$15, \$22}';" Traceback (most recent call last): File "C:\temp\procfs-snapshot\snapshot.py", line 145, in main(parse_args()) File "C:\temp\procfs-snapshot\snapshot.py", line 136, in main system_stats, processes, memory_stats = read_stats(args) ^^^^^^^^^^^^^^^^ File "C:\temp\procfs-snapshot\snapshot.py", line 115, in read_stats stats = read_tailed_files(stdout) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\temp\procfs-snapshot\parsers\tail.py", line 79, in read_tailed_files _parse_section(section_name, current_process, current_thread, data, out) File "C:\temp\procfs-snapshot\parsers\tail.py", line 44, in _parse_section _save_stat(current_thread, out['stat'])

KeyError: 'stat'

The sql database is created, but no data is in it:
<img width="892" alt="image" src="https://github.com/darrengarvey/procfs-snapshot/assets/1056405/a86086de-86c1-4710-a90e-49772ae11d08">

Any guidance appreciated.

[darrengarvey]

@MikeKav great suggestions.

1. If you put the suggested changes in a PR I will review them and you get to keep your name on the fixes.

2. I'd suggest virtualenv, conda, pyenv, etc. to easily switch between python versions.

3. This tool is based on procfs, which is a Linux/Unix/BSD/Mac API for seeing memory info. Windows may have that today but it didn't last time I checked and I'm guessing that's what you have? That said, it looks like cygwin has procfs so this may work in that environment too.

Like you suggested, some python 3 changes like print are needed, but I think these should be minor. I don't see any reason to maintain backwards compatibility with python 2 anymore.