I have two functions, g2c_open() which opens a file without an index, and g2c_read_index() which opens the file using an index.
I expected using the index to be faster, but it's slower. Here's some code from tst_index.c:
printf("Testing speed of g2c_read_index() on file %s downloaded via FTP...\n", FTP_FILE);
{
int g2cid;
int num_msg;
struct timeval start_time, end_time, diff_time;
int open_1_us, open_2_us;
int ret;
/* Open the data file without using the index file. */
if (gettimeofday(&start_time, NULL))
return G2C_ERROR;
if ((ret = g2c_open(FTP_FILE, 0, &g2cid)))
return ret;
if (gettimeofday(&end_time, NULL))
return G2C_ERROR;
if (nc4_timeval_subtract(&diff_time, &end_time, &start_time))
return G2C_ERROR;
open_1_us = (int)diff_time.tv_sec * MILLION + (int)diff_time.tv_usec;
/* Check some stuff. */
if ((ret = g2c_inq(g2cid, &num_msg)))
return ret;
if (num_msg != 688)
return G2C_ERROR;
/* Close the file. */
if ((ret = g2c_close(g2cid)))
return ret;
/* Open the data file using the index file. */
if (gettimeofday(&start_time, NULL))
return G2C_ERROR;
if ((ret = g2c_read_index(FTP_FILE, REF_FTP_FILE, 0, &g2cid)))
return ret;
if (gettimeofday(&end_time, NULL))
return G2C_ERROR;
if (nc4_timeval_subtract(&diff_time, &end_time, &start_time))
return G2C_ERROR;
open_2_us = (int)diff_time.tv_sec * MILLION + (int)diff_time.tv_usec;
printf("open without index %d with index %d, using index saved %d microseconds\n",
open_1_us, open_2_us, open_1_us - open_2_us);
/* Check some stuff. */
if ((ret = g2c_inq(g2cid, &num_msg)))
return ret;
if (num_msg != 688)
return G2C_ERROR;
/* Close the file. */
if ((ret = g2c_close(g2cid)))
return ret;
This produces the following output:
Testing speed of g2c_read_index() on file WW3_Regional_US_West_Coast_20220718_0000.grib2 downloaded via FTP...
open without index 8235 with index 22059, using index saved -13824 microseconds
Part of the problem ,may be that the index file is missing one important piece of information, the section length for section 7. This is a vital number for producing the output of degrib2, but it is not stored in the index file. So when opening with an index file, I also have to open the data file and read the section lengths for every section 7. Perhaps this slows things down.
It would be good to run a profiler and find the detailed explanation for why opening with an index is slower.
I have two functions, g2c_open() which opens a file without an index, and g2c_read_index() which opens the file using an index.
I expected using the index to be faster, but it's slower. Here's some code from tst_index.c:
This produces the following output:
Part of the problem ,may be that the index file is missing one important piece of information, the section length for section 7. This is a vital number for producing the output of degrib2, but it is not stored in the index file. So when opening with an index file, I also have to open the data file and read the section lengths for every section 7. Perhaps this slows things down.
It would be good to run a profiler and find the detailed explanation for why opening with an index is slower.