szcompressor / SZ

Error-bounded Lossy Data Compressor (for floating-point/integer datasets)
http://szcompressor.org
Other
155 stars 56 forks source link

How to run SZ API code #114

Open clodiamery opened 1 year ago

clodiamery commented 1 year ago

Hello, I install SZ in this way

  1. mkdir build && cd build
  2. cmake .. -DCMAKE_INSTALL_PREFIX:PATH=[INSTALL_DIR]
  3. make
  4. make install Then cd example . I need to use SZ API in my project. If I use this command to compress the .dat file, it works correctly : ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.dat 8 8 128 Or ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.dat 8192

My problem I can't use txt file as input, if I use ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.txt 8 8 128

I got on Failed to open input file Error: file testdata/x86/testdouble_8_8_128.txt 8 8 128 cannot be read.

Now I have two question: 1- How can I use txt file as input? 2- How can run (.c) program? because I need to run (testdouble_compress.c) as .c file

Any help please? Thanks,

disheng222 commented 1 year ago

SZ command accepts only binary format files instead of txt files. In fact, txt file is not a dense format to store the data because it uses strings essentially to store that information. In other words, if you just convert all the text strings to the binary representation, you will see reduced data size.

FYI, you can use qcat to do the format conversion between the txt file and binary file. https://github.com/szcompressor/qcat After installing qcat and building it, you'll find some executables like 'convertTxtToBytesFloat' in the examples/ directory. This is what you can use to do the conversion. convertTxtToBytesFloat will read a txt file in which each line represents a data value and the output will be a binary file containing those data values in float type. Then, you can feed the binary file to SZ for a further compression.

Best, Sheng

On Fri, Aug 4, 2023 at 6:55 AM clodiamery @.***> wrote:

Hello, I install SZ in this way

1.

mkdir build && cd build

2.

cmake .. -DCMAKE_INSTALL_PREFIX:PATH=[INSTALL_DIR]

3.

make

4.

make install

Then cd example . I need to use SZ API in my project. If I use this command to compress the .dat file, it works correctly : ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.dat 8 8 128 Or ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.dat 8192

My problem I can't use txt file as input, if I use ./testdouble_compress sz.config testdata/x86/testdouble_8_8_128.txt 8 8 128

I got on Failed to open input file Error: file testdata/x86/testdouble_8_8_128.txt 8 8 128 cannot be read.

Now I have two question: 1- How can I use txt file as input? 2- How can run (.c) program? because I need to run (testdouble_compress.c) as .c file

Any help please? Thanks,

— Reply to this email directly, view it on GitHub https://github.com/szcompressor/SZ/issues/114, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACK3KSIIAIQGCCQRWKDIGQLXTTPJFANCNFSM6AAAAAA3EDHF5Q . You are receiving this because you are subscribed to this thread.Message ID: @.***>

clodiamery commented 1 year ago

Thanks Sheng, I wanted to run the code in this issue https://github.com/szcompressor/SZ/issues/93

I put the code writeCompressedDatatoTxt.c in example folder

I have no lib folder inside SZ directory why? how can build it correctly ?

disheng222 commented 2 months ago

Well. You need to modify Makefile.am. The corresponding knowledge/skill is automake (or autotool). You can learn it from google. The basic steps are as follows:

  1. modify Makefile.am by adding the following line: writeCompressedDatatoTxt_SOURCES=writeCompressedDatatoTxt.c writeCompressedDatatoTxt_LDADD=../sz/.libs/libSZ.a ../zlib/.libs/libzlib.a ../zstd/.libs/libzstd.a -lm
  2. Create an executable file called autogen.sh under the root directory of the SZ project, and copy the following lines into it:

    !/bin/bash

aclocal libtoolize -f -c autoconf autoheader automake -a (You can use the following command make it executable: chmod +x autogen.sh)

  1. Then, in the SZ root directory, run "./configure;make". You should be able to see the compiled file for writeCompressedDatatoTxt.c.