search

astrojs / fitsjs

JavaScript library for reading the FITS astronomical format. Functionality includes reading of images, data cubes, compressed images, binary and ascii tables.
http://astrojs.github.io/fitsjs/
MIT License
85 stars 21 forks source link

Handling large FITS files #16

Closed Merg1255 closed 10 years ago

Merg1255 commented 10 years ago

Would it be easy to mention up to which file size can the js library read data? For example, can it handle 1GB files inside a browser through HTML5 chunk loading? Any examples?

Thanks

kapadia commented 10 years ago

@Merg1255 The max file size depends on the browser. Firefox is better able to allocated large blocks of memory, partly due to better memory management of typed arrays, and partly because it's a 64 bit application. Chrome struggles when allocating large blocks of memory, and needs chunk loading.

Depending on the browser, you may be able to handle > 1GB files. Extra trickery is needed is loading a file over XHR.

Merg1255 commented 10 years ago

Good to know. I think preparing an example with a public available big fits file would be great.

Merg1255 commented 10 years ago

@kapadia What are the remaining functionalities that could also be implemented in the library? The Readme.md has what's supported, but what else should there be in it? For example, will fits.js read all fits files, or are there any exceptions to this?

kapadia commented 10 years ago

@Merg1255 This library reads all flavors of FITS files (images, volumes, tables, etc), however there are often cases when astro pipelines don't write FITS according to specification. In those cases, reading might fail. Are you working with this library or are you interested in some of the techniques used in development?

Merg1255 commented 10 years ago

My colleagues were planning to build something with this library, so I have to check first that it will work as intended, at least with the normal fits files. Could you please try to resolve the other issues I've filed, as well as include your contact information?

Thanks!

kapadia commented 10 years ago

@Merg1255 I'm curious on what you'd like to build. Can you share? Other issues are resolved, following practices borrowed from other well used js libs (e.g. license and contact).

Merg1255 commented 10 years ago

@kapadia I've updated a reply on one of the closed topics. The project will most likely depend on the needs of the current working team, after also getting feedback from other teams as to how a fits reader would benefit us inside a browser. It's not something decided yet, but you might be contacted for more info if such a projects sets place.

Thanks

kapadia commented 10 years ago

@Merg1255 Sounds good. Happy to chat if needed :)

Merg1255 commented 10 years ago

Thanks!