phong-phuong
commented
4 years ago Finally got Minecraft properly installed.
There are two issues, one is one or more arguments used in FFmpegFrameRecorder cause it to error upon starting. Height not divible by 2?
org.bytedeco.javacv.FrameRecorder$Exception: avcodec_open2() error -542398533: Could not open video codec. at org.bytedeco.javacv.FFmpegFrameRecorder.startUnsafe(FFmpegFrameRecorder.java:743) at org.bytedeco.javacv.FFmpegFrameRecorder.start(FFmpegFrameRecorder.java:401) at org.deeplearning4j.rl4j.util.VideoRecorder.startRecording(VideoRecorder.java:92) at org.deeplearning4j.rl4j.learning.HistoryProcessor.startMonitor(HistoryProcessor.java:70) at org.deeplearning4j.rl4j.util.DataManagerTrainingListener.onNewEpoch(DataManagerTrainingListener.java:44) at org.deeplearning4j.rl4j.learning.listener.TrainingListenerList.notifyNewEpoch(TrainingListenerList.java:71) at org.deeplearning4j.rl4j.learning.sync.SyncLearning.train(SyncLearning.java:91) at rl4j.MalmoPixels.malmoCliffWalk(MalmoPixels.java:125) at l4j.MalmoPixels.main(MalmoPixels.java:69) 20:39:30.296 [main] INFO org.deeplearning4j.malmo.MalmoEnv - Waiting for the mission to start java.lang.IllegalArgumentException: Invalid size: cannot get size of dimension 2 for rank 1 NDArray (array shape: [230400]) at org.nd4j.linalg.api.ndarray.BaseNDArray.size(BaseNDArray.java:4510) at org.datavec.image.loader.NativeImageLoader.asMat(NativeImageLoader.java:799) at org.datavec.image.loader.NativeImageLoader.asFrame(NativeImageLoader.java:775) at org.deeplearning4j.rl4j.util.VideoRecorder.record(VideoRecorder.java:113) at org.deeplearning4j.rl4j.learning.HistoryProcessor.record(HistoryProcessor.java:96) at org.deeplearning4j.rl4j.util.LegacyMDPWrapper.record(LegacyMDPWrapper.java:145) at org.deeplearning4j.rl4j.util.LegacyMDPWrapper.reset(LegacyMDPWrapper.java:111) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.refacInitMdp(QLearning.java:158) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.trainEpoch(QLearning.java:94) at org.deeplearning4j.rl4j.learning.sync.SyncLearning.train(SyncLearning.java:96) at rl4j.MalmoPixels.malmoCliffWalk(MalmoPixels.java:125) at rl4j.MalmoPixels.main(MalmoPixels.java:69)
Second is the issue I described earlier in the origial post.
Exception in thread "main" java.lang.IllegalArgumentException: Invalid size: cannot get size of dimension 2 for rank 1 NDArray (array shape: [230400]) at org.nd4j.linalg.api.ndarray.BaseNDArray.size(BaseNDArray.java:4510) at org.datavec.image.loader.NativeImageLoader.asMat(NativeImageLoader.java:799) at org.datavec.image.loader.NativeImageLoader.asFrame(NativeImageLoader.java:775) at org.deeplearning4j.rl4j.observation.transform.legacy.EncodableToImageWritableTransform.transform(EncodableToImageWritableTransform.java:43) at org.deeplearning4j.rl4j.observation.transform.legacy.EncodableToImageWritableTransform.transform(EncodableToImageWritableTransform.java:37) at org.deeplearning4j.rl4j.observation.transform.TransformProcess.transform(TransformProcess.java:106) at org.deeplearning4j.rl4j.util.LegacyMDPWrapper.reset(LegacyMDPWrapper.java:118) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.refacInitMdp(QLearning.java:158) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.trainEpoch(QLearning.java:94) at org.deeplearning4j.rl4j.learning.sync.SyncLearning.train(SyncLearning.java:96) at rl4j.MalmoPixels.malmoCliffWalk(MalmoPixels.java:125) at rl4j.MalmoPixels.main(MalmoPixels.java:69) [libx264 @ 0000000137a32940] -qscale is ignored, -crf is recommended. [libx264 @ 0000000137a32940] height not divisible by 2 (4x427)
Deeplearning4j version: 1.0.0-beta-7
Issue Description:
Exception in thread "main" java.lang.IllegalArgumentException: Invalid size: cannot get size of dimension 2 for rank 1 NDArray
The observation "MalmoBox" is a rank 1 array of double while QLearningDiscreteConv expects a rank 3 array.
So that was any fix, I created a custom observation class for game pixel data,and passed a rank 3 array in the form of [x y pixels channels, 1, 1] - that's the only configuration that work, but then I get an out of bounds error.
Rank: 3, DataType: DOUBLE, Offset: 0, Order: c, Shape: [1200,1,1], Stride: [1,1,1] Exception in thread "main" java.lang.IndexOutOfBoundsException: 176 at org.bytedeco.javacpp.indexer.Indexer.checkIndex(Indexer.java:94) at org.bytedeco.javacpp.indexer.UByteRawIndexer.put(UByteRawIndexer.java:95) at org.bytedeco.javacpp.indexer.UByteRawIndexer.put(UByteRawIndexer.java:119) at org.bytedeco.javacpp.indexer.UByteIndexer.putDouble(UByteIndexer.java:146) at org.bytedeco.javacpp.indexer.UByteIndexer.putDouble(UByteIndexer.java:33) at org.datavec.image.loader.NativeImageLoader.asMat(NativeImageLoader.java:846) at org.datavec.image.loader.NativeImageLoader.asFrame(NativeImageLoader.java:775) at org.deeplearning4j.rl4j.observation.transform.legacy.EncodableToImageWritableTransform.transform(EncodableToImageWritableTransform.java:43) at org.deeplearning4j.rl4j.observation.transform.legacy.EncodableToImageWritableTransform.transform(EncodableToImageWritableTransform.java:37) at org.deeplearning4j.rl4j.observation.transform.TransformProcess.transform(TransformProcess.java:106) at org.deeplearning4j.rl4j.util.LegacyMDPWrapper.reset(LegacyMDPWrapper.java:118) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.refacInitMdp(QLearning.java:158) at org.deeplearning4j.rl4j.learning.sync.qlearning.QLearning.trainEpoch(QLearning.java:94) at org.deeplearning4j.rl4j.learning.sync.SyncLearning.train(SyncLearning.java:96)