search

SpectacularAI / sdk

Spectacular AI SDK
https://www.spectacularai.com/
71 stars 12 forks source link

'SpectacularAI ERROR: x:441 #31

Closed GGuussss closed 7 months ago

GGuussss commented 7 months ago

Hello, I'm trying to use sai-cli process with data from Android phone generated with SpectacutalAI app, but in any case it returned 'SpectacularAI ERROR: x:441 Abandon (core dumped)'

I have activated my nerfstudio conda environment nerfstudio is working well, and i have done pip install spectacularAI[full]

I can't find anywhere anything that help me

I tried this on 3 different machines on : Laptop i7-8850 RAM 32GB + Quadro P3200 6 GB (windows) Desktop AMD FX6200 RAM 16GB + RTX3060 12GB (windows) Server Xeon(R) W-2245 RAM 128GB + 2x RTX6000 24GB (linux)

What can i do ?

oseiskar commented 7 months ago

This sounds strange. Can you perhaps post an example recording (the zip file) where this happens here? Does it happen with all recordings or just some of them? If you don't want to post the full data, could you post the files: calibration.json, and vio_config.yaml?

Also: what version of the SDK is used? (run this to check)

pip freeze | grep spectacularAI
GGuussss commented 7 months ago

thank for your answer

Created with SpectacularAI app on LineageOS 2020240213-microG-pro1x on FX-tech Pro1X phone

pip freeze | grep spectacularAI spectacularAI==1.31.0

here the command line : with the full message

$ sai-cli process ../data/raw/data5/ ../data/inputs/data5/
Selected device type: android
{'maxMapSize': 0, 'useSlam': True, 'passthroughColorImages': True, 'keyframeDecisionDistanceThreshold': 0.05, 'icpVoxelSize': 0.05, 'useStereo': False, 'maxKeypoints': 1000, 'optimizerMaxIterations': 30, 'parameterSets': ['wrapper-base', 'offline-base', 'android']}
SpectacularAI ERROR: x:441
Abandon (core dumped)

Can my "non standard" phone generate wrong data causing the issue ?

Is there anyway a set of data from android, in order to test the process ?

GGuussss commented 7 months ago

Ok my phone seems to be the problem, i tried with a colleague phone and there is no error.

But now, what's causing the problem ?

oseiskar commented 7 months ago

That's a good question. Can't unfortunately say for sure without the video file, since there was nothing apparently wrong with vio_config.yaml nor calibration.json.

My first guess would be HW video encoding, which your phone may not support very well. Is the video file in the recording package viewable on your computer? Does the situation change if you disable the "HW video encoding" flag in the settings view of the app?

You could also try updating the Android app to the latest version (released a few days ago), but this is probably not the root cause of the problem.

GGuussss commented 7 months ago

Hi, here a link with a video that doesn't work : https://filetransfer.io/data-package/q2SPqZC4#link

GGuussss commented 7 months ago

With HW encode disabled no x441 error, but mapping failled

Selected device type: android {'maxMapSize': 0, 'useSlam': True, 'passthroughColorImages': True, 'keyframeDecisionDistanceThreshold': 0.05, 'icpVoxelSize': 0.05, 'useStereo': False, 'maxKeypoints': 1000, 'optimizerMaxIterations': 30, 'parameterSets': ['wrapper-base', 'offline-base', 'android']} SpectacularAI WARN: Discarding a bad frame 631 of camera 0. SpectacularAI WARN: Discarding a bad frame 632 of camera 0. SpectacularAI WARN: Discarding a bad frame 633 of camera 0. SpectacularAI WARN: Discarding a bad frame 634 of camera 0. SpectacularAI WARN: Discarding a bad frame 635 of camera 0. SpectacularAI WARN: Discarding a bad frame 636 of camera 0. SpectacularAI WARN: Discarding a bad frame 637 of camera 0. SpectacularAI WARN: Discarding a bad frame 638 of camera 0. SpectacularAI WARN: Discarding a bad frame 639 of camera 0. SpectacularAI WARN: Discarding a bad frame 640 of camera 0. SpectacularAI WARN: Discarding a bad frame 641 of camera 0. SpectacularAI WARN: Discarding a bad frame 642 of camera 0. SpectacularAI WARN: Discarding a bad frame 643 of camera 0. SpectacularAI WARN: Discarding a bad frame 644 of camera 0. SpectacularAI WARN: Discarding a bad frame 645 of camera 0. SpectacularAI WARN: Discarding a bad frame 646 of camera 0. SpectacularAI WARN: Discarding a bad frame 647 of camera 0. SpectacularAI WARN: Discarding a bad frame 648 of camera 0. SpectacularAI WARN: Discarding a bad frame 649 of camera 0. SpectacularAI WARN: Discarding a bad frame 650 of camera 0. SpectacularAI WARN: Discarding a bad frame 651 of camera 0. SpectacularAI WARN: Discarding a bad frame 652 of camera 0. SpectacularAI WARN: Discarding a bad frame 653 of camera 0. SpectacularAI WARN: Discarding a bad frame 654 of camera 0. SpectacularAI WARN: Discarding a bad frame 655 of camera 0. SpectacularAI WARN: Discarding a bad frame 656 of camera 0. SpectacularAI WARN: Discarding a bad frame 657 of camera 0. SpectacularAI WARN: Discarding a bad frame 658 of camera 0. SpectacularAI WARN: Discarding a bad frame 659 of camera 0. SpectacularAI WARN: Discarding a bad frame 660 of camera 0. SpectacularAI WARN: Discarding a bad frame 661 of camera 0. SpectacularAI WARN: Discarding a bad frame 662 of camera 0. SpectacularAI WARN: Discarding a bad frame 663 of camera 0. SpectacularAI WARN: Discarding a bad frame 664 of camera 0. SpectacularAI WARN: Discarding a bad frame 665 of camera 0. SpectacularAI WARN: Discarding a bad frame 666 of camera 0. SpectacularAI WARN: Discarding a bad frame 667 of camera 0. SpectacularAI WARN: Discarding a bad frame 668 of camera 0. SpectacularAI WARN: Discarding a bad frame 669 of camera 0. SpectacularAI WARN: Discarding a bad frame 670 of camera 0. SpectacularAI WARN: Discarding a bad frame 671 of camera 0. SpectacularAI WARN: Discarding a bad frame 672 of camera 0. SpectacularAI WARN: Discarding a bad frame 673 of camera 0. SpectacularAI WARN: Discarding a bad frame 674 of camera 0. SpectacularAI WARN: Discarding a bad frame 675 of camera 0. SpectacularAI WARN: Discarding a bad frame 676 of camera 0. SpectacularAI WARN: Discarding a bad frame 677 of camera 0. SpectacularAI WARN: Discarding a bad frame 678 of camera 0. SpectacularAI WARN: Discarding a bad frame 679 of camera 0. SpectacularAI WARN: Discarding a bad frame 680 of camera 0. SpectacularAI WARN: Discarding a bad frame 681 of camera 0. SpectacularAI WARN: Discarding a bad frame 682 of camera 0. SpectacularAI WARN: Discarding a bad frame 683 of camera 0. SpectacularAI WARN: Discarding a bad frame 684 of camera 0. SpectacularAI WARN: Discarding a bad frame 685 of camera 0. SpectacularAI WARN: Discarding a bad frame 686 of camera 0. SpectacularAI WARN: Discarding a bad frame 687 of camera 0. SpectacularAI WARN: Discarding a bad frame 688 of camera 0. SpectacularAI WARN: Discarding a bad frame 689 of camera 0. SpectacularAI WARN: Discarding a bad frame 690 of camera 0. SpectacularAI WARN: Discarding a bad frame 691 of camera 0. SpectacularAI WARN: Discarding a bad frame 692 of camera 0. SpectacularAI WARN: Discarding a bad frame 693 of camera 0. SpectacularAI WARN: Discarding a bad frame 694 of camera 0. SpectacularAI WARN: Discarding a bad frame 695 of camera 0. SpectacularAI WARN: Discarding a bad frame 696 of camera 0. SpectacularAI WARN: Discarding a bad frame 697 of camera 0. SpectacularAI WARN: Discarding a bad frame 698 of camera 0. SpectacularAI WARN: Discarding a bad frame 699 of camera 0. SpectacularAI WARN: Discarding a bad frame 700 of camera 0. SpectacularAI WARN: Discarding a bad frame 701 of camera 0. SpectacularAI WARN: Discarding a bad frame 702 of camera 0. SpectacularAI WARN: Discarding a bad frame 703 of camera 0. SpectacularAI WARN: Discarding a bad frame 704 of camera 0. SpectacularAI WARN: Discarding a bad frame 705 of camera 0. SpectacularAI WARN: Discarding a bad frame 706 of camera 0. SpectacularAI WARN: Discarding a bad frame 707 of camera 0. SpectacularAI WARN: Discarding a bad frame 708 of camera 0. SpectacularAI WARN: Discarding a bad frame 709 of camera 0. SpectacularAI WARN: Discarding a bad frame 710 of camera 0. SpectacularAI WARN: Discarding a bad frame 711 of camera 0. SpectacularAI WARN: Discarding a bad frame 712 of camera 0. SpectacularAI WARN: Discarding a bad frame 713 of camera 0. SpectacularAI WARN: Discarding a bad frame 714 of camera 0. SpectacularAI WARN: Discarding a bad frame 715 of camera 0. SpectacularAI WARN: Discarding a bad frame 716 of camera 0. SpectacularAI WARN: Discarding a bad frame 717 of camera 0. SpectacularAI WARN: Discarding a bad frame 718 of camera 0. SpectacularAI WARN: Discarding a bad frame 719 of camera 0. SpectacularAI WARN: Discarding a bad frame 720 of camera 0. SpectacularAI WARN: Discarding a bad frame 721 of camera 0. SpectacularAI WARN: Discarding a bad frame 722 of camera 0. SpectacularAI WARN: Discarding a bad frame 723 of camera 0. SpectacularAI WARN: Discarding a bad frame 724 of camera 0. SpectacularAI WARN: Discarding a bad frame 725 of camera 0. SpectacularAI WARN: Discarding a bad frame 726 of camera 0. SpectacularAI WARN: Discarding a bad frame 727 of camera 0. SpectacularAI WARN: Discarding a bad frame 728 of camera 0. SpectacularAI WARN: Discarding a bad frame 729 of camera 0. SpectacularAI WARN: Discarding a bad frame 730 of camera 0. SpectacularAI WARN: Discarding a bad frame 731 of camera 0. SpectacularAI WARN: Discarding a bad frame 732 of camera 0. SpectacularAI WARN: Discarding a bad frame 733 of camera 0. SpectacularAI WARN: Discarding a bad frame 734 of camera 0. SpectacularAI WARN: Discarding a bad frame 735 of camera 0. SpectacularAI WARN: Discarding a bad frame 736 of camera 0. SpectacularAI WARN: Discarding a bad frame 737 of camera 0. SpectacularAI WARN: Discarding a bad frame 738 of camera 0. SpectacularAI WARN: Discarding a bad frame 739 of camera 0. SpectacularAI WARN: Discarding a bad frame 740 of camera 0. SpectacularAI WARN: Discarding a bad frame 741 of camera 0. SpectacularAI WARN: Discarding a bad frame 742 of camera 0. SpectacularAI WARN: Discarding a bad frame 743 of camera 0. SpectacularAI WARN: Discarding a bad frame 744 of camera 0. SpectacularAI WARN: Discarding a bad frame 745 of camera 0. SpectacularAI WARN: Discarding a bad frame 746 of camera 0. SpectacularAI WARN: Discarding a bad frame 747 of camera 0. SpectacularAI WARN: Discarding a bad frame 748 of camera 0. SpectacularAI WARN: Discarding a bad frame 749 of camera 0. SpectacularAI WARN: Discarding a bad frame 750 of camera 0. SpectacularAI WARN: Discarding a bad frame 751 of camera 0. SpectacularAI WARN: Discarding a bad frame 752 of camera 0. SpectacularAI WARN: Discarding a bad frame 753 of camera 0. SpectacularAI WARN: Discarding a bad frame 754 of camera 0. SpectacularAI WARN: Discarding a bad frame 755 of camera 0. SpectacularAI WARN: Discarding a bad frame 756 of camera 0. Mapping failed: no output generated

On player video can be ready without problem

GGuussss commented 7 months ago

Here is something i did not understand ! I captured a video from the app, in hardware mode (container mp4 - codecH264), and with VLC, I reencode it into with the codec used in software mode (container avi - codec MJPEG) same error x:441 I captured a video from the app, in software mode (container avi - codec MJPG), and with VLC, I reencode it into with the codec used in software mode (container mp4 - codec H264) bad frames error.

I don't understand ... reencode the video change nothing.

I looked at the files vio_config.yaml, in software mode le last parameter is not the same

HW mode

parameterSets: [wrapper-base,live,fast-vio,android]
useSlam: False
useStereo: False
fullPointCloud: True
skipNonDepthFramesInRecording: True

SW mode :

parameterSets: [wrapper-base,live,fast-vio,android]
useSlam: False
useStereo: False
fullPointCloud: True
passthroughColorImages: True

I also invert both videos (HW mode with SW mode files(json jsonl yaml)) the HW mode always generate error x:441

EDIT:

ok, I think I have found a problem, "long videos" generate the error x:441 ! 56s@30fps is not working

how many frames Sai-cli process can get ?

oseiskar commented 7 months ago

Hi. We now checked your data. Actually, the problem was in the calibration.json file, which shows that your phone returned incorrect camera calibration data. In particular it has zero principal point ("principalPointX":0,"principalPointY":0, should actually be relatively close to image center).

So the problem is that Lineage OS has not implemented all the necessary features in the Android Camera 2 API correctly for your phone model, which does not surprise me at all, since many of the commercial vendors do not implement it correctly either.

If you want to spend a lot of time hacking this, the correct fix making sure this data is populated correctly for your phone (either return null or w/2, h/2 as the principal point, not zeros).

EDIT: in any case, thank you for reporting, we will clarify the error message in the next (or other near-future) SDK release :+1:

GGuussss commented 7 months ago

Ok thank you for the informations !