Blosc / c-blosc2

A fast, compressed, persistent binary data store library for C.
https://www.blosc.org
Other
447 stars 83 forks source link

Error using metalayers #73

Open aleixalcacer opened 5 years ago

aleixalcacer commented 5 years ago

@FrancescAlted, When I tried to use the metalayers in blosc I get an error.

The example below consists of:

If the metalayers lines are commented it works well. But if the metalayer is added, when it tries to append a buffer, blosc fails.

#include <blosc2.h>

int main() {

    blosc2_frame *frame = blosc2_new_frame("metalayer.blosc2");

    blosc2_schunk *sc = blosc2_new_schunk(BLOSC_CPARAMS_DEFAULTS, BLOSC_DPARAMS_DEFAULTS, frame);

    char name[] = "metalayer";
    uint64_t content = 1234;
    uint32_t content_size = sizeof(uint64_t);
    blosc2_frame_add_metalayer(sc->frame, name, (uint8_t *) &content, content_size);

    uint64_t *content2;
    uint32_t content_size2;
    blosc2_frame_get_metalayer(sc->frame, name, (uint8_t **) &content2, &content_size2);

    printf("Content %llu\n", *content2);

    size_t part_size = 100 * 100 * sc->typesize;

    uint8_t *part = malloc(part_size);

    for (int i = 0; i < 2; ++i) {
        int err = blosc2_schunk_append_buffer(sc, part, part_size);
        printf("Error %d\n", err);
    }

    free(part);

    return 0;
}
FrancescAlted commented 5 years ago

I can reproduce this. The problem is that the mechanism for synchronizing the metalayers between schunks and frames is far from perfect. As a workaround, you need to create the frame first, add metalayers and create the schunk last. Something like this works:

#include <blosc2.h>

int main() {

  blosc2_frame *frame = blosc2_new_frame("metalayer.blosc2");

  char name[] = "metalayer";
  uint64_t content = 1234;
  uint32_t content_size = sizeof(uint64_t);
  blosc2_frame_add_metalayer(frame, name, (uint8_t *) &content, content_size);

  uint64_t *content2;
  uint32_t content_size2;
  blosc2_frame_get_metalayer(frame, name, (uint8_t **) &content2, &content_size2);
  printf("Content %llu\n", *content2);

  blosc2_schunk *sc = blosc2_new_schunk(BLOSC_CPARAMS_DEFAULTS, BLOSC_DPARAMS_DEFAULTS, frame);

  size_t part_size = 100 * 100 * sc->typesize;

  uint8_t *part = malloc(part_size);

  for (int i = 0; i < 2; ++i) {
    int err = blosc2_schunk_append_buffer(sc, part, part_size);
    printf("Error %d\n", err);
  }

  free(part);

  return 0;
}