search

troosh / openjpeg

Automatically exported from code.google.com/p/openjpeg
Other
0 stars 0 forks source link

openjpeg does not support large files #103

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
See discussion on ML :
https://groups.google.com/forum/#!topic/openjpeg/zGAIlZgxePE

Changeset 1091 to allow support for files between 2 and 4 GB :
http://code.google.com/p/openjpeg/source/detail?r=1091

Original issue reported on code.google.com by julien.m...@c-s.fr on 24 Nov 2011 at 4:37

GoogleCodeExporter commented 9 years ago 
Good pointer about the subject :
http://ac-archive.sourceforge.net/largefile/index.html

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 7:41

GoogleCodeExporter commented 9 years ago 
See also discussion from September 2011:
https://groups.google.com/forum/#!topic/openjpeg/gbmI4o6ICoM

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 7:41

GoogleCodeExporter commented 9 years ago 
See GraphicsMagick implementation for autoconf :
http://graphicsmagick.hg.sourceforge.net/hgweb/graphicsmagick/graphicsmagick/fil
e/a64b0369522f/configure.ac#l378

Support in autoconf seems to be based on the following two macros :
AC_SYS_LARGEFILE
AC_FUNC_SEEKO

AC_SYS_LARGEFILE enables the necessary #define to ensure off_t is 64 bits
The typical program to compile for that is :

#include <sys/types.h>
 /* Check that off_t can represent 2**63 - 1 correctly.
    We can't simply define LARGE_OFF_T to be 9223372036854775807,
    since some C++ compilers masquerading as C compilers
    incorrectly reject 9223372036854775807.  */
#define LARGE_OFF_T (((off_t) 1 << 62) - 1 + ((off_t) 1 << 62))
  int off_t_is_large[(LARGE_OFF_T % 2147483629 == 721
               && LARGE_OFF_T % 2147483647 == 1)
              ? 1 : -1];
int
main ()
{

  ;
  return 0;
}

Tests are run with :
- nothing
- _FILE_OFFSET_BITS 64
-  _LARGE_FILES

Then the fseeko function availability is tested with AC_FUNC_SEEKO

The following program is tested :
#include <sys/types.h> /* for off_t */
#include <stdio.h>
int
main ()
{
int (*fp) (FILE *, off_t, int) = fseeko;
     return fseeko (stdin, 0, 0) && fp (stdin, 0, 0);
  ;
  return 0;
}

Tests are run with :
- nothing
- _LARGEFILE_SOURCE macro

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 7:54

GoogleCodeExporter commented 9 years ago 
Then, results are used :

1- to redefine fseek and ftell :
http://graphicsmagick.hg.sourceforge.net/hgweb/graphicsmagick/graphicsmagick/fil
e/a64b0369522f/magick/studio.h#l98

2 - add an additional abstraction layer to handle windows API :
http://graphicsmagick.hg.sourceforge.net/hgweb/graphicsmagick/graphicsmagick/fil
e/a64b0369522f/magick/studio.h#l388

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 8:00

GoogleCodeExporter commented 9 years ago 
CMake does not seem to provide a built in equivalent module.

Some examples can be found trying to mimic this, like
http://julien.isonoe.net/git/internships/2011-lig/gmxis/plain/modules/gromacs/cm
ake/gmxTestLargeFiles.cmake

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 8:05

GoogleCodeExporter commented 9 years ago 
Based on this, changeset 1091 was wrong.

Compilation was done with large file macros activated inherited from somewhere 
else, which actually activated the 64 bit api, and made fseek resolve to the 64 
bits version, thus making it possible to reach the OPJ_UINT32 limit. My bad.

Changeset reverted in rev1094.

Original comment by julien.m...@c-s.fr on 27 Nov 2011 at 8:32

GoogleCodeExporter commented 9 years ago 
Related changesets implementing Large File support :

1097
1098
1099
1100
1104
1108
1110
1111
1113

Original comment by julien.m...@gmail.com on 1 Dec 2011 at 2:00

GoogleCodeExporter commented 9 years ago

Original comment by julien.m...@c-s.fr on 1 Dec 2011 at 2:02