gauteh / dars

𓃢 A fast and light OPeNDAP server
MIT License
20 stars 4 forks source link

use stable rust #37

Closed alexander-buerger-met-no closed 3 months ago

alexander-buerger-met-no commented 3 months ago
alexander-buerger-met-no commented 3 months ago

@magnusuMET I am not sure how to avoid the make_extents function in dars/src/lib.rs. This function is copied from the hidefix tests for Extents.

magnusuMET commented 3 months ago

I don't see an easy way since type inference through the result type is tricky. You might make stream_xdr take an Extents instead, pushing the trouble of making Extents upwards in the stack

magnusuMET commented 3 months ago

The memory leak is from H5MM_xstrdup, found by running

env RUSTFLAGS="-Z sanitizer=address" ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 cargo test --target x86_64-unknown-linux-gnu -p dars --test coads -- --nocapture coads_subset
gauteh commented 3 months ago

The memory leak is from H5MM_xstrdup, found by running

env RUSTFLAGS="-Z sanitizer=address" ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 cargo test --target x86_64-unknown-linux-gnu -p dars --test coads -- --nocapture coads_subset

Interesting, probably of interest to hdf5 as well.

magnusuMET commented 3 months ago

We run the address sanitizer in the hdf5 crate which does not reveal any leaks. The backtrace isn't very helpful either and I am not sure how to track down the leak

gauteh commented 3 months ago

Maybe it is caused by dars, some of the attribute stuff is using hdf directly.

ons. 14. aug. 2024, 11:01 skrev Magnus Ulimoen @.***>:

We run the address sanitizer in the hdf5 crate which does not reveal any leaks. The backtrace isn't very helpful either and I am not sure how to track down the leak

— Reply to this email directly, view it on GitHub https://github.com/gauteh/dars/pull/37#issuecomment-2288228332, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN3654MHJF6CUFWNKWKVLZRMMFPAVCNFSM6AAAAABMONX3IOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBYGIZDQMZTGI . You are receiving this because you commented.Message ID: @.***>

magnusuMET commented 3 months ago

Reading data/dmrpp/unused/ntypes.h5 gives the memory leak, in specific the compound_type_4 dataset

gauteh commented 3 months ago

Reading data/dmrpp/unused/ntypes.h5 gives the memory leak, in specific the compound_type_4 dataset

Those files were found in the hyrax test suite. They also appear to be less unused than the path suggests.

magnusuMET commented 3 months ago

I removed that troublesome file. If someone wants to hunt down that leak we can always add it back in.

Clippy was also really strict for package metadata, if we are planning on releasing the crates on crates.io we could blacklist the warnings again.

gauteh commented 3 months ago

Great! Thanks a lot both!