search

Esri / i3s-spec

This repository hosts the specification for Scene Layers which are containers for arbitrarily large amounts of geographic data. The delivery and persistence model for Scene Layers, referred to as Indexed 3d Scene Layer (I3S) and Scene Layer Package (SLPK) respectively, are specified.
Other
318 stars 85 forks source link

i3s converter extract to cloud blob storage will hang #96

Closed tom62291 closed 3 years ago

tom62291 commented 3 years ago

Hi, sometimes the i3s converter extract process will hang when the destination is cloud storage.

It just hanged like below and the progress will not go anymore.

image

rvargasESRI commented 3 years ago

Thanks for bringing this up @tom62291 Can you provide a sample of the data you are trying to extract to cloud storage?

Also, which cloud service are you trying to extract to?

tom62291 commented 3 years ago

It is a large integrated mesh SLPK around 110 GB and I am using Azure blob storage.

Sorry that I can not provide sample data due to NDA.

rvargasESRI commented 3 years ago

@tom62291
While testing, we have seen similar behavior when credentials are incorrect and when the container is incorrect. Can you please verify that you have write permissions to the container, that the container exists, and the correct credentials are being used.

tom62291 commented 3 years ago

The credential should be correct as I can see that 1134124 files in the blob container

rvargasESRI commented 3 years ago

@tom62291 Does your extraction hang indefinitely (and you cancel) or do you eventually receive an output?

tom62291 commented 3 years ago

It hang indefinitely

tom62291 commented 3 years ago

any news? if no, can provide any guidelines to unpack the SLPK to i3srest which stored in the local drive first, and then use azcopy command provided by Microsoft to upload the local i3srest folder to azure blob storage?

rvargasESRI commented 3 years ago

@tom62291 We believe this could be a network issue. How many threads are being used to process the SLPK to Azure? If you are using multiple threads, can you try using a single thread? Does the converter get stuck at the same amount completed when using a single thread?

There is no easy way to perform unpacking locally. You could extract to ESLPK and then would have to rename all the file names to conform to I3SRest format.

rvargasESRI commented 3 years ago

@tom62291 We have made updates to the i3s_converter which hopefully addressed these issues.