search

BYU-PCCL / holodeck

High Fidelity Simulator for Reinforcement Learning and Robotics Research.
https://holodeck.cs.byu.edu
MIT License
572 stars 42 forks source link

MazeWorld dies after a while #66

Closed joshgreaves closed 5 years ago

joshgreaves commented 5 years ago

I was running the maze world with 10k episodes of 600 timesteps. On the 80th episode the game window shut down and my program hung. More complete logging could help get to the bottom of this, and there was no error message.

joshgreaves commented 5 years ago

Even with shorted episodes, it still crashes at 80. Maybe something to do with calling reset 80 times?

joshgreaves commented 5 years ago

Just tried with european forest, crashed at episode 70

joshgreaves commented 5 years ago

From log file:

[2018.08.08-20.45.58:126][ 72]LogHolodeck: HolodeckGameMode initialized
[2018.08.08-20.45.58:176][ 72]LogWorld: Bringing World /Game/EuropeanForest/Maps/EuropeanForest.EuropeanForest up for play (max tick
 rate 0) at 2018.08.08-14.45.58
[2018.08.08-20.45.58:179][ 72]LogTemp: Warning: UAV Controller Initialized
[2018.08.08-20.45.58:179][ 72]LogHolodeck: Pawn Possessed: UAVBlueprint_128, Controlled by: HolodeckUAVController
[2018.08.08-20.45.58:179][ 72]LogWorld: Bringing up level for play took: 0.049731
[2018.08.08-20.45.58:179][ 72]LogGameMode: FindPlayerStart: PATHS NOT DEFINED or NO PLAYERSTART with positive rating
[2018.08.08-20.45.58:179][ 72]LogHolodeck: HolodeckGameMode starting play
[2018.08.08-20.45.58:179][ 72]LogHolodeck: Registering Settings
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Reset signal registered
[2018.08.08-20.45.58:180][ 72]LogHolodeck: CommandCenter::CommandCenter() constructed
[2018.08.08-20.45.58:180][ 72]LogHolodeck: CommandCenter:: is getting command buffer
[2018.08.08-20.45.58:180][ 72]LogGameMode: Display: Match State Changed from EnteringMap to WaitingToStart
[2018.08.08-20.45.58:180][ 72]LogGameState: Match State Changed from EnteringMap to WaitingToStart
[2018.08.08-20.45.58:180][ 72]LogGameMode: Display: Match State Changed from WaitingToStart to InProgress
[2018.08.08-20.45.58:180][ 72]LogTemp: Warning: Getting buffer of size 24
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor IMUSensor for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor IMUSensor for agent uav0
[2018.08.08-20.45.58:180][ 72]LogTemp: Warning: Getting buffer of size 12
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor LocationSensor for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor LocationSensor for agent uav0
[2018.08.08-20.45.58:180][ 72]LogTemp: Warning: Getting buffer of size 36
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor OrientationSensor for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor OrientationSensor for agent uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: UHolodeckCamera::BeginPlay
[2018.08.08-20.45.58:180][ 72]LogTemp: Warning: Getting buffer of size 262144
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor PixelCamera for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor PixelCamera for agent uav0
[2018.08.08-20.45.58:180][ 72]LogTemp: Warning: Getting buffer of size 12
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor VelocitySensor for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor VelocitySensor for agent uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Initializing HolodeckAgent
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor Reward for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor Reward for agent uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor Terminal for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing sensor Terminal for agent uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing action space for uav0
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing action space for uav0_teleport_bool
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Subscribing action space for uav0_teleport_command
[2018.08.08-20.45.58:180][ 72]LogHolodeck: Buffer name: uav0_hyperparameter
[2018.08.08-20.45.58:210][ 72]LogLinux: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to write memory at address 0x00000000ffffffff

[2018.08.08-20.45.58:210][ 72]LogLinux: Fatal error!

[Callstack]  00  0x000000000249ff8f  FLinuxPlatformStackWalk::CaptureStackBackTrace(unsigned long long*, unsigned int, void*)
[Callstack]  01  0x000000000236ec45  FGenericPlatformStackWalk::StackWalkAndDump(char*, unsigned long, int, void*)
[Callstack]  02  0x0000000002451e70  FLinuxCrashContext::CaptureStackTrace()
[Callstack]  03  0x00000000057b9e4b  CommonLinuxCrashHandler(FGenericCrashContext const&)
[Callstack]  04  0x00000000024533f3  PlatformCrashHandler(int, siginfo_t*, void*)
[Callstack]  05  0x00007f16cd4cd890  /lib/x86_64-linux-gnu/libpthread.so.0(+0x12890) [0x7f16cd4cd890]
[Callstack]  06  0x00007f16cc61ec14  /lib/x86_64-linux-gnu/libc.so.6(+0x18ec14) [0x7f16cc61ec14]
[Callstack]  07  0x0000000002319cd4  AUAV::SetHyperparameterAddress(float*)
[Callstack]  08  0x000000000231e110  AHolodeckPawnController::GetBuffers(FString const&)
[Callstack]  09  0x000000000231c42a  AHolodeckAgent::InitializeController()
[Callstack]  10  0x00000000023165fb  AHolodeckAgent::BeginPlay()
[Callstack]  11  0x0000000002318cec  AUAV::BeginPlay()
[Callstack]  12  0x00000000043677ad  AActor::DispatchBeginPlay()
[Callstack]  13  0x000000000513ac9a  AWorldSettings::NotifyBeginPlay()
[Callstack]  14  0x000000000488b03d  AGameMode::HandleMatchHasStarted()
[Callstack]  15  0x000000000488b744  AGameMode::SetMatchState(FName)
[Callstack]  16  0x000000000231f194  AHolodeckGameMode::StartPlay()
[Callstack]  17  0x000000000512ab49  UWorld::BeginPlay()
[Callstack]  18  0x0000000005053c00  UEngine::LoadMap(FWorldContext&, FURL, UPendingNetGame*, FString&)
[Callstack]  19  0x000000000504fbf0  UEngine::Browse(FWorldContext&, FURL, FString&)
[Callstack]  20  0x0000000005050f9c  UEngine::TickWorldTravel(FWorldContext&, float)
[Callstack]  21  0x000000000485c71d  UGameEngine::Tick(float, bool)
[Callstack]  22  0x0000000002301ce4  FEngineLoop::Tick()
[Callstack]  23  0x000000000230599a  GuardedMain(wchar_t const*)
[Callstack]  24  0x00000000057baac4  CommonLinuxMain(int, char**, int (*)(wchar_t const*))
[Callstack]  25  0x00007f16cc4b1b97  /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f16cc4b1b97]
[Callstack]  26  0x00000000022f756e  /home/josh/.local/share/holodeck/worlds/LinuxDefaultWorlds/LinuxNoEditor/Holodeck/Binaries/Linux/Holodeck() [0x22f756e]
joshgreaves commented 5 years ago

Same as #74 Fixed in LowLevelOverhaul