search

PhotonVision / photonvision

PhotonVision is the free, fast, and easy-to-use computer vision solution for the FIRST Robotics Competition.
https://photonvision.org
GNU General Public License v3.0
260 stars 169 forks source link

Import Camera config error: content too large #1275

Open clrozeboom opened 4 months ago

clrozeboom commented 4 months ago

Describe the bug We calibrated one of our cameras using 100+ images and ended up with a 63MB file when we exported the settings. We'd like to be able to restore this calibration if needed, but we get an error saying the Content is too large.

Wondering if there is a way to calibrate this larger, by hand, or if it can be updated in a hotfix? Are there alternative ways to upload the data that we could employ? Thanks!

To Reproduce You can see the file we're importing and the log here: https://github.com/FRC5010/FRCLibrary/tree/Crescendo2024/PhotonVision This is a pretty standard Arducam Global Shutter camera.

Platform:

Additional context

Mar 11 21:55:43 photonvision java[714]: [JettyServerThreadPool-41] WARN io.javalin.Javalin - Body greater than max size (50000000 bytes)
Mar 11 21:55:43 photonvision java[714]: [2024-03-11 21:55:43] [WebServer - RequestHandler] [ERROR] An error occurred while uploading calibration data: Content Too Large
Mar 11 21:55:43 photonvision java[714]: [2024-03-11 21:55:43] [WebServer - RequestHandler] [ERROR] io.javalin.http.HttpResponseException: Content Too Large
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContextKt.throwContentTooLargeIfContentTooLarge(JavalinServletContext.kt:178)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.Context.bodyAsBytes(Context.kt:126)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContext.access$bodyAsBytes$s-1678783089(JavalinServletContext.kt:34)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContext$body$2.invoke(JavalinServletContext.kt:97)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContext$body$2.invoke(JavalinServletContext.kt:97)
Mar 11 21:55:43 photonvision java[714]:         at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContext.getBody(JavalinServletContext.kt:97)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServletContext.bodyAsBytes(JavalinServletContext.kt:98)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.Context.body(Context.kt:117)
Mar 11 21:55:43 photonvision java[714]:         at org.photonvision.server.RequestHandler.onDataCalibrationImportRequest(RequestHandler.java:518)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.routing.HandlerEntry.handle(HandlerEntry.kt:19)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.DefaultTasks.HTTP$lambda$8$lambda$6$lambda$5(DefaultTasks.kt:39)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServlet.handleTask(JavalinServlet.kt:88)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServlet.handleSync(JavalinServlet.kt:53)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.http.servlet.JavalinServlet.service(JavalinServlet.kt:41)
Mar 11 21:55:43 photonvision java[714]:         at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.jetty.JavalinJettyServlet.service(JavalinJettyServlet.kt:58)
Mar 11 21:55:43 photonvision java[714]:         at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:587)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:764)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:529)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1570)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:221)
Mar 11 21:55:43 photonvision java[714]:         at io.javalin.jetty.JettyServer$start$wsAndHttpHandler$1.doHandle(JettyServer.kt:61)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:176)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:484)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1543)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:174)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1303)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:129)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:173)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:122)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.Server.handle(Server.java:563)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.HttpChannel.lambda$handle$0(HttpChannel.java:505)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:762)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:497)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:282)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:314)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:100)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.io.SelectableChannelEndPoint$1.run(SelectableChannelEndPoint.java:53)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.runTask(AdaptiveExecutionStrategy.java:416)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.consumeTask(AdaptiveExecutionStrategy.java:385)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:272)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.lambda$new$0(AdaptiveExecutionStrategy.java:140)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:411)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
Mar 11 21:55:43 photonvision java[714]:         at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
Mar 11 21:55:43 photonvision java[714]:         at java.base/java.lang.Thread.run(Thread.java:829)
Mar 11 21:55:43 photonvision java[714]: 
Mar 11 21:55:43 photonvision java[714]: [2024-03-11 21:55:43] [WebServer - Server] [DEBUG] Handled HTTP request of type POST from endpoint /api/calibration/importFromData for host 10.50.10.194 in 16.426361 ms
mcm001 commented 4 months ago

Yeah, there's a 50MB upload limit set in Server.java. You could increase this limit by changing the code, or you can strip images from the calibration JSON using https://github.com/PhotonVision/photonvision/pull/1244

clrozeboom commented 4 months ago

That looks promising, is that implemented? If so, could you point to documentation or explain how to use it? Thanks! Edit - is it the python extraction process talked about here? https://docs.photonvision.org/en/latest/docs/calibration/calibration.html#accessing-calibration-images

5010TigerDynasty commented 4 months ago

Hey, when trying to use the calibrationUtils.py, it requires a package for mrcal. How can that be installed?

clrozeboom commented 4 months ago

This is as far as I was able to get with the calibrationUtils.py utility after installing it on my photonvision server.

python3 calibrationUtils.py photon_calibration_Arducam_OV9281_USB_Camera_1280x720.json camera_cal/ Traceback (most recent call last): File "/home/pv/calibrationUtils.py", line 255, in main() File "/home/pv/calibrationUtils.py", line 251, in main convert_photon_to_mrcal(args.input, args.output_folder) File "/home/pv/calibrationUtils.py", line 228, in convert_photon_to_mrcal mrcal_model = convert_cal_to_mrcal_cameramodel(camera_cal_data) File "/home/pv/calibrationUtils.py", line 135, in convert_cal_to_mrcal_cameramodel [ File "/home/pv/calibrationUtils.py", line 139, in ).reshape((cal.calobjectSize.width, cal.calobjectSize.height, 3)) TypeError: 'float' object cannot be interpreted as an integer

clrozeboom commented 4 months ago

I think we are going to redo our calibration with only 60+ images to cut down the size of the export. Being able to have a restorable calibration for this level of effort in calibrating is crucial.

mcm001 commented 4 months ago

You can also go in and replace the observations array in the json with an empty list. I would accept a PR increasing the max upload size as well.

And yeah looks like a bug in the script in the PR you posted above. Should be super easy to just cast to int?

mcm001 commented 4 months ago

You could also patch the frontend to clear observation snapshot data if the file size is above 50Mb, that's super easy too