Data Ingestion Component

geosolutions-it / sciadro-backend

https://waffle.io/geosolutions-it/sciadro-backend

1 stars 3 forks source link

Data Ingestion Component #4

Closed tdipisa closed 5 years ago

tdipisa commented 5 years ago

Mission files to ingest in the model are:

.avi (the video will be uploaded and stored by the backend in a pluggable way, ready to be served on demand)
.telem (custom format and contains only the useful telemetry information)
.xml (generated by a specific routine in post processing after the mission or in real time during the mission video shooting)

A python backend service needs to be developed to ingest mission data.

sergey-misuk-yotec commented 5 years ago

Video and telem files upload is going to be handled using built-in framework functionality.

Points considered to be the same probably don't have the same coordinates, so some threshold must be introduced for which points are considered to be the same. which point should be chosen as a result? Do we need to calculate some average value?

tdipisa commented 5 years ago

@sergey-misuk-yotec

Video and telem files upload is going to be handled using built-in framework functionality.

ok, but the upload is part of another issue #2 :-)

Points considered to be the same probably don't have the same coordinates, so some threshold must be introduced for which points are considered to be the same. which point should be chosen as a result? Do we need to calculate some average value?

This point needs to be investigated to see if we need to do further processing for performance purposes. I think the best approach is to initially ingest all data and evaluate them better then we can evaluate if further processing is needed, how and where: first we need to have a clear overview of the data before considerng further optimizations, I will open a separated issue for this (#7)

simboss commented 5 years ago

@sergey-misuk-yotec I agree with @tdipisa

I would open a separate issue (as an investigation for the input points clean up).

That said, can you elaborate more on the approach you have in mind for the ingestion? Ideally this would run asynchronously also with respect to the upload.

Aside, some care must be taken for the fact that these points are 3D since the drone flies and sometimes it might simply go up and down. Postgis can handle 3D geometries AFAIK.