Windows builds

[niphlod]

Is a windows build considered ? I know, everything is not MS rocks... but it seems I can't build this on a Win machine.

[chriso]

I haven't tried and haven't got a windows machine to test with unfortunately. If a build fails and you provide the logs I may be able to help.

[niphlod]

Ok, ASAP I'll retry a build and attach the logs here. Thanks for the fast reply ^_^

[niphlod]

I'm not an expert in packaging and building, anyway here's the output....

>>> python setup.py install
running install
running build
running build_py
running build_ext
building 'libgauged' extension
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/array.c -o build\temp
.win32-2.7\Release\lib\array.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/hash.c -o build\temp.
win32-2.7\Release\lib\hash.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/sort.c -o build\temp.
win32-2.7\Release\lib\sort.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/map.c -o build\temp.w
in32-2.7\Release\lib\map.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/writer.c -o build\tem
p.win32-2.7\Release\lib\writer.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
writing build\temp.win32-2.7\Release\lib\libgauged.def
c:\DevKit\mingw\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\lib\array.o build\temp.win32-2.7\Release\lib\hash.o
build\temp.win32-2.7\Release\lib\sort.o build\temp.win32-2.7\Release\lib\map.o build\temp.win32-2.7\Release\lib\writer.o
 build\temp.win32-2.7\Release\lib\libgauged.def -Lc:\python27\libs -Lc:\python27\PCbuild -lpython27 -lmsvcr90 -o build\l
ib.win32-2.7\libgauged.pyd
Cannot export initlibgauged: symbol not defined
build\temp.win32-2.7\Release\lib\sort.o:sort.c:(.text+0x341): undefined reference to `pthread_create'
build\temp.win32-2.7\Release\lib\sort.o:sort.c:(.text+0x38e): undefined reference to `pthread_create'
build\temp.win32-2.7\Release\lib\sort.o:sort.c:(.text+0x3a2): undefined reference to `pthread_join'
build\temp.win32-2.7\Release\lib\sort.o:sort.c:(.text+0x3b6): undefined reference to `pthread_join'
collect2.exe: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

[chriso]

I've done some research and it looks I'll need to do the following

• #ifdef out the #include <pthread> and mergesort implementation when compiling under Windows. I think an #if defined(_WIN32) || defined(WIN32) is the typical way to accomplish this.
• Define a GAUGED_EXPORT which is equal to __declspec(dllexport) when compiling under Windows (blank otherwise), and then prefix function definitions with it, e.g. GAUGED_EXPORT gauged_array_t *gauged_array_new() { ... }.
• Update the shared library glob to look for *.{so,dll} rather than just *.so.

ctypes should handle the rest.

[chriso]

K, I've created a windows branch. Could you possibly try compiling and let me know if there's any issues?

[niphlod]

ok, I think it's a step forward. This is the error now

running install
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\gauged
copying gauged\aggregates.py -> build\lib.win32-2.7\gauged
copying gauged\bridge.py -> build\lib.win32-2.7\gauged
copying gauged\config.py -> build\lib.win32-2.7\gauged
copying gauged\context.py -> build\lib.win32-2.7\gauged
copying gauged\gauged.py -> build\lib.win32-2.7\gauged
copying gauged\lru.py -> build\lib.win32-2.7\gauged
copying gauged\utilities.py -> build\lib.win32-2.7\gauged
copying gauged\version.py -> build\lib.win32-2.7\gauged
copying gauged\writer.py -> build\lib.win32-2.7\gauged
copying gauged\__init__.py -> build\lib.win32-2.7\gauged
creating build\lib.win32-2.7\gauged\errors
copying gauged\errors\__init__.py -> build\lib.win32-2.7\gauged\errors
creating build\lib.win32-2.7\gauged\results
copying gauged\results\statistics.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\time_series.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\__init__.py -> build\lib.win32-2.7\gauged\results
creating build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\interface.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\mysql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\postgresql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\sqlite.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\__init__.py -> build\lib.win32-2.7\gauged\drivers
creating build\lib.win32-2.7\gauged\structures
copying gauged\structures\float_array.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\sparse_map.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\__init__.py -> build\lib.win32-2.7\gauged\structures
running build_ext
building 'libgauged' extension
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\lib
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/array.c -o build\temp
.win32-2.7\Release\lib\array.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/hash.c -o build\temp.
win32-2.7\Release\lib\hash.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/sort.c -o build\temp.
win32-2.7\Release\lib\sort.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/map.c -o build\temp.w
in32-2.7\Release\lib\map.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
lib/map.c:45:33: error: external linkage required for symbol 'gauged_map_resize' because of 'dllexport' attribute
error: command 'gcc' failed with exit status 1

[chriso]

I accidentally exported a static function. Try again now.

[chriso]

By the way, once you've built it successfully you can run the test suite with

python test.py --quick

[niphlod]

ok, now map compiles successfully but something is still going bad

running install
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\gauged
copying gauged\aggregates.py -> build\lib.win32-2.7\gauged
copying gauged\bridge.py -> build\lib.win32-2.7\gauged
copying gauged\config.py -> build\lib.win32-2.7\gauged
copying gauged\context.py -> build\lib.win32-2.7\gauged
copying gauged\gauged.py -> build\lib.win32-2.7\gauged
copying gauged\lru.py -> build\lib.win32-2.7\gauged
copying gauged\utilities.py -> build\lib.win32-2.7\gauged
copying gauged\version.py -> build\lib.win32-2.7\gauged
copying gauged\writer.py -> build\lib.win32-2.7\gauged
copying gauged\__init__.py -> build\lib.win32-2.7\gauged
creating build\lib.win32-2.7\gauged\errors
copying gauged\errors\__init__.py -> build\lib.win32-2.7\gauged\errors
creating build\lib.win32-2.7\gauged\results
copying gauged\results\statistics.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\time_series.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\__init__.py -> build\lib.win32-2.7\gauged\results
creating build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\interface.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\mysql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\postgresql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\sqlite.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\__init__.py -> build\lib.win32-2.7\gauged\drivers
creating build\lib.win32-2.7\gauged\structures
copying gauged\structures\float_array.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\sparse_map.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\__init__.py -> build\lib.win32-2.7\gauged\structures
running build_ext
building 'libgauged' extension
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\lib
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/array.c -o build\temp
.win32-2.7\Release\lib\array.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/hash.c -o build\temp.
win32-2.7\Release\lib\hash.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/sort.c -o build\temp.
win32-2.7\Release\lib\sort.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/map.c -o build\temp.w
in32-2.7\Release\lib\map.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/writer.c -o build\tem
p.win32-2.7\Release\lib\writer.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
writing build\temp.win32-2.7\Release\lib\libgauged.def
c:\DevKit\mingw\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\lib\array.o build\temp.win32-2.7\Release\lib\hash.o
build\temp.win32-2.7\Release\lib\sort.o build\temp.win32-2.7\Release\lib\map.o build\temp.win32-2.7\Release\lib\writer.o
 build\temp.win32-2.7\Release\lib\libgauged.def -Lc:\python27\libs -Lc:\python27\PCbuild -lpython27 -lmsvcr90 -o build\l
ib.win32-2.7\libgauged.pyd
Cannot export initlibgauged: symbol not defined
collect2.exe: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

[chriso]

Ok, getting closer. Can you try again? It looks like you need to prefix the extension name with an underscore or else it looks for an init<library_name> symbol.

[niphlod]

this is the output now

running install
running build
running build_py
creating build
creating build\lib.win32-2.7
creating build\lib.win32-2.7\gauged
copying gauged\aggregates.py -> build\lib.win32-2.7\gauged
copying gauged\bridge.py -> build\lib.win32-2.7\gauged
copying gauged\config.py -> build\lib.win32-2.7\gauged
copying gauged\context.py -> build\lib.win32-2.7\gauged
copying gauged\gauged.py -> build\lib.win32-2.7\gauged
copying gauged\lru.py -> build\lib.win32-2.7\gauged
copying gauged\utilities.py -> build\lib.win32-2.7\gauged
copying gauged\version.py -> build\lib.win32-2.7\gauged
copying gauged\writer.py -> build\lib.win32-2.7\gauged
copying gauged\__init__.py -> build\lib.win32-2.7\gauged
creating build\lib.win32-2.7\gauged\errors
copying gauged\errors\__init__.py -> build\lib.win32-2.7\gauged\errors
creating build\lib.win32-2.7\gauged\results
copying gauged\results\statistics.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\time_series.py -> build\lib.win32-2.7\gauged\results
copying gauged\results\__init__.py -> build\lib.win32-2.7\gauged\results
creating build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\interface.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\mysql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\postgresql.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\sqlite.py -> build\lib.win32-2.7\gauged\drivers
copying gauged\drivers\__init__.py -> build\lib.win32-2.7\gauged\drivers
creating build\lib.win32-2.7\gauged\structures
copying gauged\structures\float_array.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\sparse_map.py -> build\lib.win32-2.7\gauged\structures
copying gauged\structures\__init__.py -> build\lib.win32-2.7\gauged\structures
running build_ext
building '_gauged' extension
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\lib
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/array.c -o build\temp
.win32-2.7\Release\lib\array.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/hash.c -o build\temp.
win32-2.7\Release\lib\hash.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/sort.c -o build\temp.
win32-2.7\Release\lib\sort.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/map.c -o build\temp.w
in32-2.7\Release\lib\map.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
c:\DevKit\mingw\bin\gcc.exe -mdll -O -Wall -Iinclude -Ic:\python27\include -Ic:\python27\PC -c lib/writer.c -o build\tem
p.win32-2.7\Release\lib\writer.o -O3 -std=c99 -pedantic -Wall -Wextra -pthread
writing build\temp.win32-2.7\Release\lib\_gauged.def
c:\DevKit\mingw\bin\gcc.exe -shared -s build\temp.win32-2.7\Release\lib\array.o build\temp.win32-2.7\Release\lib\hash.o
build\temp.win32-2.7\Release\lib\sort.o build\temp.win32-2.7\Release\lib\map.o build\temp.win32-2.7\Release\lib\writer.o
 build\temp.win32-2.7\Release\lib\_gauged.def -Lc:\python27\libs -Lc:\python27\PCbuild -lpython27 -lmsvcr90 -o build\lib
.win32-2.7\_gauged.pyd
Cannot export init_gauged: symbol not defined
collect2.exe: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

[chriso]

Hmm ok. I've added a dummy init_gauged symbol. Try again now.

[niphlod]

yes!!!! it builds successfully now. and this is the result of the tests

>>> python test.py --quick
..........................................................................................................
----------------------------------------------------------------------
Ran 106 tests in 0.369s

OK

[chriso]

Awesome :) I just published a version 0.4.0 with Windows support.

[niphlod]

happy to have helped somehow. BTW: benchmark.py uses the resource module that is unavailable on windows (and doesn't have an "alternative"). I didn't go through all the code to see if it's used in other places, but I'm planning to dig in to create a MSSQL adapter... I'll report any other problems popping up.

[chriso]

K I've removed the RSS usage check from the benchmarks so it should work now - it wasn't showing much anyway.

[niphlod]

sorry to bother. 0.4.0 installs fine, but then this pops up in Linux. All is fine in Windows.

>>> import gauged
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/__init__.py", line 7, in <module>
    from .gauged import Gauged
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/gauged.py", line 10, in <module>
    from .writer import Writer
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/writer.py", line 13, in <module>
    from .structures import SparseMap
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/structures/__init__.py", line 7, in <module>
    from .sparse_map import SparseMap
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/structures/sparse_map.py", line 10, in <module>
    from ..bridge import Gauged, MapPtr, Uint32Ptr, FloatPtr
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/bridge.py", line 80, in <module>
    Gauged = SharedLibrary('_gauged', 'gauged')
  File "/home/niphlod/Scrivania/venvs/gautests/local/lib/python2.7/site-packages/gauged/bridge.py", line 19, in __init__
    lib = glob.glob(shared_lib)[0]
IndexError: list index out of range

[chriso]

Does a make clean fix it? I think this occurs when there's a build directory but the shared library isn't there (because of an error, etc.)

[niphlod]

Uhm. make clean didn't fix it. I just popped up a clean ubuntu 14, did pip install gauged and the same error popped up in the python interpreter as soon as I try to "import gauged". pip remove gauged pip install gauged==0.3.0 and "import gauged" works fine.

[chriso]

I introduced a regression in 8e36865651604bd03563b4a5df12149e87837c9d which has has been fixed in 0.4.1.