search

ngageoint / sarpy

A basic Python library to demonstrate reading, writing, display, and simple processing of complex SAR data using the NGA SICD standard.
MIT License
248 stars 85 forks source link

Update tests with URLs of missing test data #154

Closed RubeRad closed 3 years ago

RubeRad commented 3 years ago

Right now just cloning the repo and 'setup.py test' fails all over the place. The error messages make clear where the test data is expected to be found, but not where a user can download it from in order to put it there.

The link in the README about the sample ntf is helpful, but not complete. I also had to hunt on my own to find required test data at

and I still can't find data for the Sentinel and RS2 tests.

It would be good if each test failure message would say like "The data needed for this test can be downloaded from http://..."

Even better, how about if there were a 'setup.py download' that automatically downloaded and organized all the required test data? (with a confirmation like 'This will download X gigabytes of test data into ...\Desktop\sarpy_testing, is that OK? [Y/n]' )

thomasmccullough commented 3 years ago

Hi RubeRad, thanks for your participation. Without some internal hosting capability which we do not currently have, and some of the data may potentially have licensing issues, the idea of an automatic download is probably infeasible.

You are certainly correct that the information needs to be updated, and more helpful messages logged. I will also ensure that tests which cannot be performed do not result in a failure.

RubeRad commented 3 years ago

Yeah I just meant downloading from whatever external URLs the data hopefully lives at, but a trail of breadcrumbs in the failure messages is a start.

Are the RS2/Sentinel data online that you know of?

thomasmccullough commented 3 years ago

I was able to find sentinel data from alaska/nasa data provider: https://search.asf.alaska.edu/#/?resultsLoaded=false&zoom=3&center=-97.493959,39.672786&view=equitorial&productTypes=SLC

You should be able to get access to some, but you have to request and account or something - it's been awhile since I did it. I've honestly never tried to find open radarsat-2 data.

To be completely clear, these unit tests are really just for me as a developer for constructing the various reader types and making sure that I don't break anything. You really don't typically need to run these as a user - there are no configuration issues or anything. Obviously, you are more than welcome to do so, however.

thomasmccullough commented 3 years ago

You can get some sample data for radarsat-2 here: https://mdacorporation.com/geospatial/international/satellites/RADARSAT-2/sample-data/ The complex format data is the SLC data.

I should say that this isn't exactly what I used for the unit tests, so it's probably not hugely helpful for this exact purpose.

RubeRad commented 3 years ago

Awesome, thx for the pointers, I will check those out and see if I am able to use them to get 11/11 on the tests.

If you want, you can drop me a ping to my same username at gmail; I'm happy to discuss my affiliation and business with sarpy offline.

thomasmccullough commented 3 years ago

I updated the structure of the tests, and put a read me in the tests directory. It's now easier to configure, at least. I'm closing this for now, but this remains a work in progress.