search

kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.
Other
219 stars 76 forks source link

Move CI to github actions #93

Closed nritsche closed 3 years ago

jrs65 commented 3 years ago

It seems like the windows build is the problem in here. I think we should just remove that until we know how to configure it properly.

james-s-willis commented 3 years ago

@nritsche what is the state of this PR? Can I help out on this? Or is it just a matter of removing the windows build?

nritsche commented 3 years ago

@nritsche what is the state of this PR? Can I help out on this? Or is it just a matter of removing the windows build?

Thanks! Now it looks like bitshuffle isn't installed properly in the macOS build

james-s-willis commented 3 years ago 
2021-05-25T16:13:05.3695980Z ============================= test session starts ==============================
2021-05-25T16:13:05.3697790Z platform darwin -- Python 3.6.13, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
2021-05-25T16:13:05.3698520Z rootdir: /Users/runner/work/bitshuffle/bitshuffle
2021-05-25T16:13:05.3699100Z collected 61 items
2021-05-25T16:13:05.3699430Z 
2021-05-25T16:13:06.4572380Z tests/test_ext.py ...................................................... [ 88%]
2021-05-25T16:13:07.5525270Z ..                                                                       [ 91%]
2021-05-25T16:13:07.7107910Z tests/test_h5filter.py FFF                                               [ 96%]
2021-05-25T16:13:07.7414850Z tests/test_h5plugin.py F                                                 [ 98%]
2021-05-25T16:13:07.7659180Z tests/test_regression.py .                                               [100%]
2021-05-25T16:13:07.7660520Z 
2021-05-25T16:13:07.7661480Z =================================== FAILURES ===================================
2021-05-25T16:13:07.7662060Z ____________________________ TestFilter.test_filter ____________________________
2021-05-25T16:13:07.7662440Z 
2021-05-25T16:13:07.7662960Z self = <test_h5filter.TestFilter testMethod=test_filter>
2021-05-25T16:13:07.7663420Z 
2021-05-25T16:13:07.7663810Z     def test_filter(self):
2021-05-25T16:13:07.7664260Z         shape = (32 * 1024 + 783,)
2021-05-25T16:13:07.7664650Z         chunks = (4 * 1024 + 23,)
2021-05-25T16:13:07.7665030Z         dtype = np.int64
2021-05-25T16:13:07.7665480Z         data = np.arange(shape[0])
2021-05-25T16:13:07.7665950Z         fname = "tmp_test_filters.h5"
2021-05-25T16:13:07.7666450Z         f = h5py.File(fname, "w")
2021-05-25T16:13:07.7666870Z         h5.create_dataset(
2021-05-25T16:13:07.7667270Z             f,
2021-05-25T16:13:07.7667600Z             b"range",
2021-05-25T16:13:07.7667990Z             shape,
2021-05-25T16:13:07.7668430Z             dtype,
2021-05-25T16:13:07.7668790Z             chunks,
2021-05-25T16:13:07.7669240Z             filter_pipeline=(32008, 32000),
2021-05-25T16:13:07.7669820Z             filter_flags=(h5z.FLAG_MANDATORY, h5z.FLAG_MANDATORY),
2021-05-25T16:13:07.7670390Z >           filter_opts=None,
2021-05-25T16:13:07.7671080Z         )
2021-05-25T16:13:07.7671330Z 
2021-05-25T16:13:07.7671750Z tests/test_h5filter.py:33: 
2021-05-25T16:13:07.7672200Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2021-05-25T16:13:07.7672820Z bitshuffle/h5.pyx:175: in bitshuffle.h5.create_dataset
2021-05-25T16:13:07.7673530Z     dset_id = h5d.create(parent.id, name, tid, sid, dcpl=dcpl)
2021-05-25T16:13:07.7674340Z h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
2021-05-25T16:13:07.7674860Z     ???
2021-05-25T16:13:07.7675410Z h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
2021-05-25T16:13:07.7675940Z     ???
2021-05-25T16:13:07.7695600Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2021-05-25T16:13:07.7695980Z 
2021-05-25T16:13:07.7696280Z >   ???
2021-05-25T16:13:07.7696830Z E   ValueError: Unable to create dataset (error during user callback)
2021-05-25T16:13:07.7697280Z 
2021-05-25T16:13:07.7697650Z h5py/h5d.pyx:76: ValueError
2021-05-25T16:13:07.7699230Z ----------------------------- Captured stderr call -----------------------------
2021-05-25T16:13:07.7701170Z HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
2021-05-25T16:13:07.7702450Z   #000: H5Pocpl.c line 1027 in H5Pget_filter_by_id2(): can't find object for ID
2021-05-25T16:13:07.7703180Z     major: Object atom
2021-05-25T16:13:07.7703750Z     minor: Unable to find atom information (already closed?)
2021-05-25T16:13:07.7706470Z   #001: H5Pint.c line 4015 in H5P_object_verify(): property list is not a member of the class
2021-05-25T16:13:07.7707540Z     major: Property lists
2021-05-25T16:13:07.7708340Z     minor: Unable to register new atom
2021-05-25T16:13:07.7708960Z   #002: H5Pint.c line 3965 in H5P_isa_class(): not a property list
2021-05-25T16:13:07.7709560Z     major: Invalid arguments to routine
2021-05-25T16:13:07.7710080Z     minor: Inappropriate type
james-s-willis commented 3 years ago

It's failing here. Do you have a mac to debug this? I'm not sure about Unable to find atom information (already closed?)

james-s-willis commented 3 years ago

Don't we have to install h5py from source for bitshuffle?

  - name: Install h5py
              if: ${{ matrix.os == 'macos-latest' }}   
              run: |   
                      pip install h5py==2.9
james-s-willis commented 3 years ago

Oh great, that looks like it fixed the build! @nritsche @jrs65

nritsche commented 3 years ago

Oh great, that looks like it fixed the build! @nritsche @jrs65

Great! I'll let someone else approve. Thanks @james-s-willis

jrs65 commented 3 years ago

Looks good to me. You might just want to rebase all the commits into something sensible @james-s-willis (e.g. there are commits for turning on and then off windows builds)