Readline package error per Python install

[holisticinfosec]

pip install pybeagle fails at readline package Collecting readline==6.2.4.1 (from pybeagle) Downloading https://files.pythonhosted.org/packages/f4/01/2cf081af8d880b44939a5f1b446551a7f8d59eae414277fd0c303757ff1b/readline-6.2.4.1.tar.gz Complete output from command python setup.py egg_info: error: this module is not meant to work on Windows

Suggest a call for pyreadline for Windows installs.

[holisticinfosec]

Specifically, Windows 10 Version 1903, Python 3.6.6, Pip 19.03

[yampelo]

I kicked off a travis job to build on windows: https://travis-ci.com/yampelo/beagle/jobs/191311316

I'm guessing this is related to deps from rekall:

$pipenv graph
rekall==1.7.1
...
  - readline [required: Any, installed: 6.2.4.1]

I'll see if bumping up the rekall version can fix it.

[holisticinfosec]

Note, and it may be a separate issue, but rekall install on Windows has breaking on Acora even with earlier Python (3.6.6) installed. Trying to debug that with isolated rekall installation, will advise.

[holisticinfosec]

Accidentally closed, reopened as such

[yampelo]

Here's what I think I'm going to do. I'm going to make rekall an extra install, that you can install by doing pip install pybeagle[rekall] or pip install pybeagle[memory]. That way the package will work on any python3+ distrib and anyone who wants to use memory will need to either leverage the docker container (the container has the library set up on it) or use a python distribution that rekall supports

[yampelo]

@holisticinfosec try pip installing the latest version now (1.0.2), simply doing pip install pybeagle --upgrade should work .It should work on windows: https://travis-ci.com/yampelo/beagle/builds/107562330

[holisticinfosec]

Better for sure, but now seeing the aforementioned acora issue creep in again. That's a rekall issue vs beagle.

building 'acora._acora' extension creating build\temp.win-amd64-3.6 creating build\temp.win-amd64-3.6\Release creating build\temp.win-amd64-3.6\Release\acora C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ic:\python36\include -Ic:\python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /Tcacora/_acora.c /Fobuild\temp.win-amd64-3.6\Release\acora/_acora.obj _acora.c c:\python36\include\pyconfig.h(222): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\bin\HostX86\x64\cl.exe' failed with exit status 2

----------------------------------------

Command "c:\python36\python.exe -u -c "import setuptools, tokenize;file='C:\Users\rmcree\AppData\Local\Temp\pip-install-3rbkf0ef\acora\setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\rmcree\AppData\Local\Temp\pip-record-r63y56xw\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\rmcree\AppData\Local\Temp\pip-install-3rbkf0ef\acora\

[yampelo]

Yeah, that one is out of my control and i noted it in https://github.com/google/rekall/issues/488, does installing without the rekall extra (pip install pybeagle vs pip install pybeagle[rekall]) work?

[holisticinfosec]

Unfortunately, pip install pybeagle[rekall] and pip install pybeagle[memory] still result in the attempted inclusion of acora and thus fail as described above.

[holisticinfosec]

After a full purge of all prior pybeagle instances, I just got a clean install via pip install pybeagle[memory]. Let's close this bug as a successful solution while we wait for google/rekall#488 to be resolved.

[yampelo]

Memory isn't a valid extra option (https://github.com/yampelo/beagle/blob/master/setup.py#L20) - only rekall is. Installing pybeagle[memory] means that you did the same as just doing pip install pybeagle, and won't be able to use the memory grapher.