search

mcaceresb / stata-parquet

Read and write parquet files from Stata
MIT License
22 stars 6 forks source link

Windows support #6

Open kylebarron opened 6 years ago

kylebarron commented 6 years ago 
git clone https://github.com/mcaceresb/stata-parquet
cd stata-parquet
conda create -n stata-parquet -c conda-forge arrow-cpp parquet-cpp boost mingw -y
conda activate stata-parquet

del src\plugin\lib
del src\plugin\spi

In file explorer, copied lib folder and spi-2.0 folders to src\plugin: image

I tried to compile with

g++ -shared -fPIC -Wall -O3 -std=c++11 -o test.plugin src/plugin/spi/stplugin.cpp src/plugin/parquet.cpp -I%CONDA_PREFIX%\include -L%CONDA_PREFIX%/lib

but to no avail...

(stata-parquet) C:\Users\Kyle\stata-parquet>g++ -shared -fPIC -Wall -O3 -std=c++11 -o test.plugin src/plugin/spi/stplugin.cpp src/plugin/parquet.cpp -I%CONDA_PREFIX%\include -L%CONDA_PREFIX%/lib
src/plugin/spi/stplugin.cpp:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
src/plugin/parquet.cpp:1:0: warning: -fPIC ignored for target (all code is position independent) [enabled by default]
In file included from src/plugin/parquet.cpp:41:0:
src/plugin/reader_writer.h:20:23: fatal error: arrow/api.h: No such file or directory
compilation terminated.
mcaceresb commented 6 years ago

What is dir %CONDA_PREFIX%\include ?

kylebarron commented 6 years ago 
(stata-parquet) C:\Users\Kyle>dir %CONDA_PREFIX%\include
 Volume in drive C has no label.
 Volume Serial Number is 3C37-1E46

 Directory of C:\Users\Kyle\Anaconda3\envs\stata-parquet\include

10/30/2018  08:19 PM    <DIR>          .
10/30/2018  08:19 PM    <DIR>          ..
06/26/2018  04:39 PM            47,451 abstract.h
06/26/2018  04:39 PM             1,016 accu.h
06/26/2018  04:39 PM             1,213 asdl.h
06/26/2018  04:39 PM               477 ast.h
06/26/2018  04:39 PM               792 bitset.h
06/26/2018  04:39 PM               264 bltinmodule.h
06/26/2018  04:39 PM               886 boolobject.h
06/26/2018  04:39 PM             2,114 bytearrayobject.h
06/26/2018  04:39 PM             8,363 bytesobject.h
06/26/2018  04:39 PM             3,191 bytes_methods.h
06/26/2018  04:39 PM               701 cellobject.h
06/26/2018  04:39 PM             8,420 ceval.h
06/26/2018  04:39 PM             1,666 classobject.h
06/26/2018  04:39 PM             5,945 code.h
06/26/2018  04:39 PM             6,793 codecs.h
06/26/2018  04:39 PM             2,164 compile.h
06/26/2018  04:39 PM             1,807 complexobject.h
06/26/2018  04:39 PM             9,414 datetime.h
06/26/2018  04:39 PM             2,964 descrobject.h
06/26/2018  04:39 PM             7,178 dictobject.h
06/26/2018  04:39 PM               458 dtoa.h
06/26/2018  04:39 PM            22,469 dynamic_annotations.h
06/26/2018  04:39 PM               253 enumobject.h
06/26/2018  04:39 PM             1,497 errcode.h
06/26/2018  04:39 PM               597 eval.h
06/26/2018  04:39 PM             1,651 fileobject.h
06/26/2018  04:39 PM             2,989 fileutils.h
06/26/2018  04:39 PM             4,794 floatobject.h
06/26/2018  04:39 PM             3,530 frameobject.h
06/26/2018  04:39 PM             4,079 funcobject.h
06/26/2018  04:39 PM             3,583 genobject.h
06/26/2018  04:39 PM             1,966 graminit.h
06/26/2018  04:39 PM             2,074 grammar.h
06/26/2018  04:39 PM             4,233 import.h
06/26/2018  04:39 PM               513 intrcheck.h
06/26/2018  04:39 PM               567 iterobject.h
06/26/2018  04:39 PM             2,900 listobject.h
06/26/2018  04:39 PM             3,761 longintrepr.h
06/26/2018  04:39 PM             8,471 longobject.h
06/26/2018  04:39 PM               803 marshal.h
06/26/2018  04:39 PM             2,765 memoryobject.h
06/26/2018  04:39 PM               253 metagrammar.h
06/26/2018  04:39 PM             3,847 methodobject.h
06/26/2018  04:39 PM             7,317 modsupport.h
06/26/2018  04:39 PM             2,285 moduleobject.h
06/26/2018  04:39 PM               349 namespaceobject.h
06/26/2018  04:39 PM             1,007 node.h
06/26/2018  04:39 PM            40,326 object.h
06/26/2018  04:39 PM            14,244 objimpl.h
06/26/2018  04:39 PM             1,288 odictobject.h
06/26/2018  04:39 PM             5,073 opcode.h
06/26/2018  04:39 PM               691 osdefs.h
06/26/2018  04:39 PM               291 osmodule.h
06/26/2018  04:39 PM             2,885 parsetok.h
06/26/2018  04:39 PM             1,128 patchlevel.h
06/26/2018  04:39 PM               253 pgen.h
06/26/2018  04:39 PM             1,180 pgenheaders.h
06/26/2018  04:39 PM             2,744 pyarena.h
06/26/2018  04:39 PM             8,128 pyatomic.h
06/26/2018  04:39 PM             1,726 pycapsule.h
06/26/2018  04:39 PM            19,743 pyconfig.h
06/26/2018  04:39 PM             1,320 pyctype.h
06/26/2018  04:39 PM             1,209 pydebug.h
06/26/2018  04:39 PM               863 pydtrace.d
06/26/2018  04:39 PM             1,970 pydtrace.h
06/26/2018  04:39 PM            17,237 pyerrors.h
06/26/2018  04:39 PM             2,340 pyexpat.h
06/26/2018  04:39 PM             8,471 pyfpe.h
06/26/2018  04:39 PM               410 pygetopt.h
06/26/2018  04:39 PM             4,139 pyhash.h
06/26/2018  04:39 PM             4,008 pylifecycle.h
06/26/2018  04:39 PM             2,989 pymacconfig.h
06/26/2018  04:39 PM             3,500 pymacro.h
06/26/2018  04:39 PM             8,312 pymath.h
06/26/2018  04:39 PM             8,491 pymem.h
06/26/2018  04:39 PM            27,573 pyport.h
06/26/2018  04:39 PM            11,146 pystate.h
06/26/2018  04:39 PM               436 pystrcmp.h
06/26/2018  04:39 PM               495 pystrhex.h
06/26/2018  04:39 PM             1,483 pystrtod.h
06/26/2018  10:35 PM            22,274 Python-ast.h
07/26/2018  04:07 AM             3,143 Python.h
06/26/2018  04:39 PM             2,974 Python.h.orig
06/26/2018  04:39 PM             6,782 pythonrun.h
06/26/2018  04:39 PM             2,992 pythread.h
06/26/2018  04:39 PM             7,609 pytime.h
06/26/2018  04:39 PM             4,307 py_curses.h
06/26/2018  04:39 PM               629 rangeobject.h
06/26/2018  04:39 PM             3,333 setobject.h
06/26/2018  04:39 PM             2,485 sliceobject.h
06/26/2018  04:39 PM             2,018 structmember.h
06/26/2018  04:39 PM             1,353 structseq.h
06/26/2018  04:39 PM             4,994 symtable.h
06/26/2018  04:39 PM             1,355 sysmodule.h
06/26/2018  04:39 PM             1,943 token.h
06/26/2018  04:39 PM             3,644 traceback.h
06/26/2018  04:39 PM             2,444 tupleobject.h
06/26/2018  04:39 PM             2,253 typeslots.h
06/26/2018  04:39 PM             1,056 ucnhash.h
06/26/2018  04:39 PM            81,768 unicodeobject.h
06/26/2018  04:39 PM             1,693 warnings.h
06/26/2018  04:39 PM             2,866 weakrefobject.h
             102 File(s)        571,837 bytes
               2 Dir(s)  182,179,340,288 bytes free

(stata-parquet) C:\Users\Kyle>
mcaceresb commented 6 years ago

So it doesn't have the arrow headers there for some reason...

kylebarron commented 6 years ago

I don't know too much about debugging Windows; just thought it would be helpful to start an issue and document what I tried

mcaceresb commented 6 years ago

Windows can be quite annoying. Will take a stab eventually, but priority is not as high.

kylebarron commented 6 years ago

Yes I agree, not a high priority.

gbekes commented 7 months ago

Huge fan of parquet, must use stata for some projects (so big thanks for this script) but I'm in Windows. any chance a windows compatible version?

mcaceresb commented 7 months ago

@gbekes No ):

I recommend you try Stata's built-in Python interface if you have 16+

gbekes commented 7 months ago

Thanks! This is for a repro package, may be an overkill.

mcaceresb commented 7 months ago

@gbekes Importing the data through Python might be less of an overkill than getting this to work on Windows. You'd just need yo read the data in Python and out it into Stata (which you can do in-memory through the interface, but if speed is not the issue you can always convert the data to .dta outside of Stata).