search

astrojs / fitsjs

JavaScript library for reading the FITS astronomical format. Functionality includes reading of images, data cubes, compressed images, binary and ascii tables.
http://astrojs.github.io/fitsjs/
MIT License
85 stars 21 forks source link

Implement getColumn for tables #3

Closed kapadia closed 10 years ago

kapadia commented 11 years ago

Get column should be implemented such that bytes are skipped appropriately to avoid unneeded calls to DataView getters methods.

Merg1255 commented 10 years ago

Are there any plans of implementation of this? Is there a clear benefit than the current form? I'd like to see the what's next on your list for this project. :)

kapadia commented 10 years ago

@Merg1255 This is an old issue that was never closed. It's implemented here:

https://github.com/astrojs/fitsjs/blob/master/src/tabular.coffee#L67

but I'd like to optimize for huge tables (> 100,000 rows). At the moment, it's inefficient because an array (as opposed to a fixed length typed array), is storing the column values.

https://github.com/astrojs/fitsjs/blob/master/src/tabular.coffee#L92

I have some ideas to improve it, but in it's current state it works well for tables with < 100,000 rows. The API will stay the same regardless of any optimizations.

Merg1255 commented 10 years ago

Good to know. I'll be watching for updates. :)

kapadia commented 10 years ago

@Merg1255 getColumn is now an order of magnitude faster. Was able to get a column containing 3.3 million rows in ~22 seconds.

Merg1255 commented 10 years ago

@kapadia that's great!! :)