search

saalfeldlab / render

Render transformed image tiles
GNU General Public License v2.0
34 stars 32 forks source link

Can you spot what goes wrong here? #115

Closed martinschorb closed 3 years ago

martinschorb commented 3 years ago

Hi,

I try to run the SIFTPointMatch Client and it fails

20/12/08 22:21:17 WARN TaskSetManager: Lost task 111.0 in stage 0.0 (TID 111, 10.11.13.175, executor 25): java.lang.IllegalStateException: failed to load data for {canvasId: '0000.0184.00451', url: 'http://pc-emcf-16.embl.de:8080/render-ws/v1/owner/SBEM/project/seaurchin_paolo_test/stack/giovanna_test0_mipmaps/tile/0000.0184.00451/render-parameters?scale=0.6&filter=true&excludeMask=true&normalizeForMatching=true', clipWidth: 500, clipHeight: 500}
    at org.janelia.alignment.match.cache.CanvasDataCache.getData(CanvasDataCache.java:179)
    at org.janelia.alignment.match.cache.CanvasDataCache.getCanvasFeatures(CanvasDataCache.java:224)
    at org.janelia.render.client.spark.SIFTPointMatchClient.lambda$generateMatchesForPairs$9e040152$1(SIFTPointMatchClient.java:219)
    at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1(JavaRDDLike.scala:102)
    at org.apache.spark.api.java.JavaRDDLike.$anonfun$mapPartitionsWithIndex$1$adapted(JavaRDDLike.scala:102)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2(RDD.scala:889)
    at org.apache.spark.rdd.RDD.$anonfun$mapPartitionsWithIndex$2$adapted(RDD.scala:889)
    at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:313)
    at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:349)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:313)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)
    at org.apache.spark.scheduler.Task.run(Task.scala:127)
    at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:444)
    at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:447)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: cannot set clip offsets for canvas 0000.0184.00451 because relative position is unknown
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2263)
    at com.google.common.cache.LocalCache.get(LocalCache.java:4000)
    at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004)
    at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874)
    at org.janelia.alignment.match.cache.CanvasDataCache.getData(CanvasDataCache.java:177)
    ... 20 more
Caused by: java.lang.IllegalArgumentException: cannot set clip offsets for canvas 0000.0184.00451 because relative position is unknown
    at org.janelia.alignment.match.CanvasId.setClipOffsets(CanvasId.java:100)
    at org.janelia.alignment.match.CanvasIdWithRenderContext.loadRenderParameters(CanvasIdWithRenderContext.java:86)
    at org.janelia.alignment.match.cache.CanvasFeatureListLoader.load(CanvasFeatureListLoader.java:128)
    at org.janelia.alignment.match.cache.CanvasFeatureListLoader.load(CanvasFeatureListLoader.java:24)
    at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599)
    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379)
    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342)
    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257)
    ... 24 more

when I run that relevant call from the submission node

curl -get "http://pc-emcf-16.embl.de:8080/render-ws/v1/owner/SBEM/project/seaurchin_paolo_test/stack/giovanna_test0_mipmaps/tile/0000.0184.00451/render-parameters?scale=0.6&filter=true&excludeMask=true&normalizeForMatching=true"

I get a valid JSON output:

{
  "meshCellSize" : 64.0,
  "minMeshCellSize" : 0.0,
  "x" : 0.0,
  "y" : 0.0,
  "width" : 2048,
  "height" : 1536,
  "scale" : 0.6,
  "areaOffset" : false,
  "convertToGray" : false,
  "quality" : 0.85,
  "numberOfThreads" : 1,
  "skipInterpolation" : false,
  "binaryMask" : false,
  "excludeMask" : true,
  "doFilter" : true,
  "addWarpFieldDebugOverlay" : false,
  "fillWithNoise" : false,
  "tileSpecs" : [ {
    "tileId" : "0000.0184.00451",
    "layout" : {
      "sectionId" : "451",
      "temca" : "3View",
      "camera" : "3View",
      "stageX" : -25417.8,
      "stageY" : -20712.266666666666,
      "rotation" : 0.0,
      "pixelsize" : 15.0
    },
    "z" : 451.0,
    "minX" : 0.0,
    "minY" : 0.0,
    "maxX" : 2048.0,
    "maxY" : 1536.0,
    "width" : 2048.0,
    "height" : 1536.0,
    "minIntensity" : 0.0,
    "maxIntensity" : 255.0,
    "mipmapLevels" : {
      "0" : {
        "imageUrl" : "file:///g/emcf/ronchi/Benvenuto-Giovanna-seaurchin_SBEMtest_5833/giovanna_G3-1_20-07-07_3view/S010_acquisition/tiles/g0000/t0184/S010_acquisition_g0000_t0184_s00451.tif"
      },
      "1" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/1/S010_acquisition_g0000_t0184_s00451.tif.png"
      },
      "2" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/2/S010_acquisition_g0000_t0184_s00451.tif.png"
      },
      "3" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/3/S010_acquisition_g0000_t0184_s00451.tif.png"
      },
      "4" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/4/S010_acquisition_g0000_t0184_s00451.tif.png"
      },
      "5" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/5/S010_acquisition_g0000_t0184_s00451.tif.png"
      },
      "6" : {
        "imageUrl" : "file:///g/emcf/schorb/render-output/mipmaps/giovanna0/6/S010_acquisition_g0000_t0184_s00451.tif.png"
      }
    },
    "transforms" : {
      "type" : "list",
      "specList" : [ ]
    },
    "meshCellSize" : 64.0
  } ],
  "minBoundsMeshCellSize" : 64.0
}

also, I checked that the tile files and mipmaps exist.

trautmane commented 3 years ago

Hi Martin,

Are you by any chance comparing cross layer tiles but using clipping (which is only intended for same layer tiles)?

The exception indicates that the tile can't be clipped because the relative position information (e.g. LEFT, RIGHT, TOP, BOTTOM) is missing from the source tile pair data - so it doesn't know which edge to clip.

--Eric

martinschorb commented 3 years ago

Hi Eric,

thanks for pointing this out. Yes, that's the reason. I gave it the wrong tilepairs.

I need to work on my wrapper to feed the parameters from a more readable source like a JSON...

Martin