NoSuchElementException when accessing list but .size() works ok

[Episodex]

Hello, I'm concatenating several videos to make one. It works for some videos (mainly rather short ones, the longest I tried with success was 5 min 56 sec total. But when I try to join longer set of videos I get exception on audio track .getSamples(). All audio tracks are cropped because of syncing issues so the are CroppedTrack in fact.

Now more into details. I'm using version 1.1.7 which I believe is the latest.

This is how error looks like in Watch:

As you can see size() works but whole list not.

The code where this happens is:

public List<Sample> getSamples() {
    return origTrack.getSamples().subList(fromSample, toSample);
}

in CroppedTrack.java. If I get into origTrack.getSamples() and run watch on private variable samples I get the error that is attached.

I think something may be wrong with the implementation of AbstractArray for cropped list. I will try to debug further to find out what's happening but maybe you'll have time to make it even faster.

The problem is with test case. It doesn't happen for all videos. I crop all of them and some are concatenated without a problem and others throw exception. The one rule I noticed so far, but can't tell it's 100% is that it doesn't work for long video sets. But maybe it's not about time but number of tracks. In this example there are 14 video tracks appended to each other. The error is thrown on second audio track. I go back to debugging. Let me know if you need more details on this.

Btw. great job with the library! I will donate for sure when this finally works for me :).

[Episodex]

I investigated this more and my best guess is that something in size() implementation in DefaultMp4SampleList.java is wrong in some circumstances:

public int size() {
    return l2i(trackBox.getSampleTableBox().getSampleSizeBox().getSampleCount());
}

Because it looks like it returns bigger size than in reality and for loop from toArray() tries to get element of index that doesn't exist. The audio track that fails for me is 5016 elements long so can be hard to check this theory but I'll try.

Maybe I messed cropping...

That's my code for cropping:

for (Track t : part.getTracks()) {
    if (t.getHandler().equals("soun")) {
        audioTrack = new CroppedTrack(t, 0, t.getSamples().size() - 12);
     }
}

[Episodex]

More info. I've catched the exception that is in fact causing OutOfRangeException that in turn is causing NoSuchElementException:

EDIT: I turned off cropping Audio tracks and still have this error. So maybe it's something wrong with the files. Do you know some common situation when this exception can occur at this place?

Any help appreciated.

[sannies]

Thank you for debugging this issue on your own that far. You are already quite close to the problem:

this method call topLevel.getByteBuffer fails with an IOException: mmap failed: EPERM (Operation not permitted).

The source of this IOException is most likely the getByteByteBuffer method of the BasicContainer

public ByteBuffer getByteBuffer(long rangeStart, long size) throws IOException { if (this.dataSource != null) { synchronized (this.dataSource) { return this.dataSource.map(this.startPosition + rangeStart, size); } }

the dataSource#map call delegates most likely to a FileDataSourceImpl and there the mmap call fails (FileDataSourceImpl#57)

return fc.map(FileChannel.MapMode.READ_ONLY, startPosition, size);

So my guess is that the fileChannel#map call fails. Do you have read access to the file?

I hope that helps you a bit with the debugging - I cannot do much as the cause for it is within your system!

2015-09-27 14:03 GMT+02:00 Episodex notifications@github.com:

More info. I've catched the exception that is in fact causing OutOfRangeException that in turn is causing NoSuchElementException: [image: error_isoparser] https://cloud.githubusercontent.com/assets/6061876/10122716/91ff9878-6520-11e5-976c-3aa64c4229e4.PNG

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-143545634.

[Episodex]

Thanks for quick answer! That's what I initially thought, that maybe I don't have the access to the file or it was deleted or something. But would the size() method return anything then? Also this fails on index 2117 out of 5016 so almost half of samples were read ok. I don't have good understanding of how mp4 format works so I don't know if it is possible to have access to only part of the file. Also video track seems to work perfect. Can it be that the file is broken or the exception then would be different or thrown earlier?

[sannies]

This 'table of content' - where I find the offsets of the actual samples - is typically in the beginning of the file and read in the beginning. Sometimes the allowed number of memory map operation is restricted per process. The first few hundred might succeed and then fail once the limit is exceeded (try setting sysctl -w vm.max_map_count=524280). Or on some Android's it's a problem to memory map parts of the file beyond 2GB.

Are you on Android? Linux? Windows?

Beste Grüße, Sebastian

2015-09-27 18:42 GMT+02:00 Episodex notifications@github.com:

Thanks for quick answer! That's what I initially thought, that maybe I don't have the access to the file or it was deleted or something. But would the size() method return anything then? Also this fails on index 2117 out of 5016 so almost half of samples were read ok. I don't have good understanding of how mp4 format works so I don't know if it is possible to have access to only part of the file. Also video track seems to work perfect. Can it be that the file is broken or the exception then would be different or thrown earlier?

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-143575873.

[Episodex]

It's Android. The file is <2GB for sure. All the files are from the same source - Phone's Camera - and are stored in the same place - DCIM directory on SD card. I'm running tests on physical phone, not emulator.

[sannies]

It's super hard to debug over the distance. If you google "Android EPERM Operation not permitted" you really get a whole bunch of questions/issues. I'll prepare a concat example with the 2.x branch (still alpha) of the mp4parser tomorrow. In 2.x I'm only using InputStreams and you won't run into those memorymap problems. Wish you a pleasant rest of the Sunday!

Beste Grüße, Sebastian

2015-09-27 19:10 GMT+02:00 Episodex notifications@github.com:

It's Android. The file is <2GB for sure. All the files are from the same source - Phone's Camera - and are stored in the same place - DCIM directory on SD card. I'm running tests on physical phone, not emulator.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-143577733.

[Episodex]

Thanks for help! I debugged it further yesterday and found out that it fails on different index each time but looks randomly to me... I'll dig into this EPERM issues on the net.

Is 2.x release matter of weeks, months, years? :)

[rniebecker]

Hey,

I just ran into the same issue on Android using 1.1.7, it worked before with 1.0-RC28. My original video is below 2GB, has a bitrate of 20mbit and 30fps.

This only happens if I append a track that is bigger than 1 min 10 secs and I call DefaultMp4Builder().build(movie). It does not happen if I append 4 tracks each 30 secs!

Here is my trace: java.util.NoSuchElementException at java.util.AbstractList$SimpleListIterator.next(AbstractList.java:59) at java.util.AbstractList$SubAbstractList$SubAbstractListIterator.next(AbstractList.java:201) at java.util.AbstractCollection.toArrayList(AbstractCollection.java:349) at java.util.AbstractCollection.toArray(AbstractCollection.java:339) at java.util.ArrayList.addAll(ArrayList.java:188) at com.googlecode.mp4parser.authoring.tracks.AppendTrack.getSamples(AppendTrack.java:372) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder$InterleaveChunkMdat.(DefaultMp4Builder.java:686) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder$InterleaveChunkMdat.(DefaultMp4Builder.java:646) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.build(DefaultMp4Builder.java:127)

I guess the workaround for now is cutting bigger tracks into smaller ones and appending them.

Cheers, Ralf

[sannies]

That is soooo weird. But good that you solved it

I'm working on the 2.x branch but it's still too alpha. I'll use 'appending' as a one of the first proof of concepts for the new API but I won't finish that usecase today.

2015-09-28 19:02 GMT+02:00 rniebecker notifications@github.com:

Hey,

I just ran into the same issue on Android using 1.1.7, it worked before with 1.0-RC28. My original video is below 2GB, has a bitrate of 20mbit and 30fps.

This only happens if I append a track that is bigger than 1 min 10 secs and I call DefaultMp4Builder().build(movie). It does not happen if I append 4 tracks each 30 secs!

Here is my trace: java.util.NoSuchElementException at java.util.AbstractList$SimpleListIterator.next(AbstractList.java:59) at java.util.AbstractList$SubAbstractList$SubAbstractListIterator.next(AbstractList.java:201) at java.util.AbstractCollection.toArrayList(AbstractCollection.java:349) at java.util.AbstractCollection.toArray(AbstractCollection.java:339) at java.util.ArrayList.addAll(ArrayList.java:188) at com.googlecode.mp4parser.authoring.tracks.AppendTrack.getSamples(AppendTrack.java:372) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder$InterleaveChunkMdat.(DefaultMp4Builder.java:686) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder$InterleaveChunkMdat.(DefaultMp4Builder.java:646) at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.build(DefaultMp4Builder.java:127)

I guess the workaround for now is cutting bigger tracks into smaller ones and appending them.

Cheers, Ralf

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-143805082.

[Episodex]

@rniebecker Thanks for input! I will try to do CroppedTracks out of longer segments then and I'll let here know if it worked.

@sannies Is there a way to know how many milliseconds one sample takes? Or is it fixed absolute value?

[sannies]

Track#getSampleDurations and track#getMetadata() #getTimeScale

Episodex notifications@github.com schrieb am Mo., 28. Sep. 2015 7:47 PM:

@rniebecker https://github.com/rniebecker Thanks for input! I will try to do CroppedTracks out of longer segments then and I'll let here know if it worked.

@sannies https://github.com/sannies Is there a way to know how many milliseconds one sample takes? Or is it fixed absolute value?

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-143818695.

[rniebecker]

Sorry guys,

this is not the solution, I just implemented it that way and sometimes it works and often it doesn't throwing the same exception. :(

Will look into the code to see if I can locate the issue.

Any chance to get my hands on the new 2.x lib early so I can test if the problem is still there?

Cheers, Ralf

[Episodex]

I used following code to segment long audiotracks:

Movie part = MovieCreator.build(dataSource);

Track videoTrack = null;
Track  audioTrack = null;

for (Track t : part.getTracks()) {
    if (t.getHandler().equals("soun")) {
        audioTrack = new CroppedTrack(t, 0, t.getSamples().size() - 12);

        long trackSize = audioTrack.getSamples().size();
        if(trackSize > 1000) {
            for(int i = 0; i * 1000 < trackSize; i++) {
                audioTracks.add(new CroppedTrack(audioTrack, i*1000, Math.min((i+1) * 1000, trackSize)));
            }
        } else {
            audioTracks.add(audioTrack);
        }
    }
    if (t.getHandler().equals("vide")) {
        videoTrack = t;
        videoTracks.add(videoTrack);
    }
}

But it doesn't work because still it iterates through original big track (probably to split them into pieces I declared, and probably inevitable :( ).

From my tests it looks like it has problem just after 1000th sample of only audio tracks.

But I just did the test and disabled audio tracks at all to leave only video tracks. Now I get "mmap failed: ENOMEM (Out of memory)" Exception on 1304th sample of a video. @sannies Silly question, but have you tested appending videos on clips longer than 1000 samples? Maybe there's some memory leak? Or it's something Android specific. But my device has 3GB of RAM...

EDIT: Just checked the memory chart and it barely popped up from 28MB to 36MB.

EDIT2: Further investigation showed that almost precisely at the moment of crash the used memory goes above free memory.

It looks like Android is not fast enough to give more memory (?). I'm by no means Android expert unfortunately. I will look into this direction on the net though.

@sannies Is it possible for you to somehow ensure enough memory for your process in your code? Or maybe I'm talking nonsense here altogether :).

[rniebecker]

Hey,

I switched back to using isoparser-1.0.7.1.jar which is working fine for me, just tested on Samsung S5 and S6.

I tested with 1.1.5 before but that failed as well. So the bug has been introduced somewhere between those versions. Would be nice to get access to versions 1.1.1 to 1.1.4 to verify where it has been introduced.

Cheers, Ralf

[Episodex]

Hi @rniebecker , can this version be used with Gradle or you had to have the .jar physically? Can this released be found somewhere? I'd do the test also on this version.

[sannies]

Hi Ralf, Episodex,

if you can would please try to use version 1.1.9 from https://oss.sonatype.org/content/repositories/comgooglecodemp4parser-1079 ?

I removed the "late mapping" memory mapping (first parse pass only recorded offsets and sizes of boxes, they were only read when properties of them where accessed ). No the content of the boxes is directly read into the heap (no worries actual sample data is not ).

the old releases have been uploaded to an S3 bucket. You should be able to use http://com.mp4parser.s3.amazonaws.com/ as repsoitory. If not simply download by hand:

http://com.mp4parser.s3.amazonaws.com/com/googlecode/mp4parser/isoparser/1.1.1/isoparser-1.1.1.jar http://com.mp4parser.s3.amazonaws.com/com/googlecode/mp4parser/isoparser/1.1.2/isoparser-1.1.2.jar http://com.mp4parser.s3.amazonaws.com/com/googlecode/mp4parser/isoparser/1.1.4/isoparser-1.1.4.jar http://com.mp4parser.s3.amazonaws.com/com/googlecode/mp4parser/isoparser/1.1.5/isoparser-1.1.5.jar http://com.mp4parser.s3.amazonaws.com/com/googlecode/mp4parser/isoparser/1.1.6/isoparser-1.1.6.jar

If the problem is potentially in 1.1.3 or 1.1.0 I can add them later. But the releases listed here were still in my local repo.

Best Regards, Sebastian

2015-09-29 13:46 GMT+02:00 Episodex notifications@github.com:

Hi @rniebecker https://github.com/rniebecker , can this version be used with Gradle or you had to have the .jar physically? Can this released be found somewhere? I'd do the test also on this version.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-144035171.

[Episodex]

Hi, thanks for uploading these versions. Unfortunately for 1.1.9 I still get out of memory exception. I will try 1.1.1 now, but I would really like to try also the version that works for @rniebecker . I'll let you know if 1.1.1 worked in a moment.

[sannies]

wait wait wait, OutOfMemory? Haven't we had an IOException with mmap failed?

2015-09-29 21:20 GMT+02:00 Episodex notifications@github.com:

Hi, thanks for uploading these versions. Unfortunately for 1.1.9 I still get out of memory exception. I will try 1.1.1 now, but I would really like to try also the version that works for @rniebecker https://github.com/rniebecker . I'll let you know if 1.1.1 worked in a moment.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-144161422.

[rniebecker]

Same here 1.1.9 is not working. Will test the other versions as well...

I don't know where I got the version 1.0.7.1 from tbh, I had it in my download folder, must have downloaded it some weeks ago.

Cheers, Ralf

[rniebecker]

Hmmm, I'm getting an access denied error when I try to download one of the other versions, looks like this:

AccessDeniedAccess Denied710A5961BFA26049DLIOmTgAkA1dNfJgeoeXqbKcMUAT1Bi3F5wRQSRwFqc/xuELLJnUjMI1DpwnyOrk/TGNVEiBLB8=

Cheers, Ralf

[Episodex]

It's "mmap failed: ENOMEM (Out of memory)" for video track after I totally removed audiotracks. I described this in detail some posts ago. Maybe one error is being triggered by another or something...

[rniebecker]

I'm still getting the NoSuchElementException with 1.1.9, nothing has changed for me.

[sannies]

Sorry, the files were not public. They are now readable for everyone.

2015-09-29 21:27 GMT+02:00 rniebecker notifications@github.com:

I'm still getting the NoSuchElementException with 1.1.9, nothing has changed for me.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-144164058.

[rniebecker]

Downloading now, thanks, will let you now ASAP ...

[Episodex]

I think we need AspectJ to run this for 1.1.1? I get an exception: "Failed resolution of: Lorg/aspectj/runtime/reflect/Factory;"

I'll try to download newest AspectJ and then try.

[rniebecker]

I have aspectj 1.8.2 in my libs folder, getting the same error for 1.1.1.:

E/AndroidRuntime(26280): java.lang.NoSuchMethodError: No interface method sort(Ljava/util/Comparator;)V in class Ljava/util/List; or its super classes (declaration of 'java.util.List' appears in /system/framework/core-libart.jar) E/AndroidRuntime(26280): at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.createStco(DefaultMp4Builder.java:476) E/AndroidRuntime(26280): at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.createStbl(DefaultMp4Builder.java:361) E/AndroidRuntime(26280): at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.createTrackBox(DefaultMp4Builder.java:322) E/AndroidRuntime(26280): at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.createMovieBox(DefaultMp4Builder.java:226) E/AndroidRuntime(26280): at com.googlecode.mp4parser.authoring.builder.DefaultMp4Builder.build(DefaultMp4Builder.java:116)

Cheers, Ralf

[Episodex]

Same here. I put aspectj-1.8.7.jar in libs and still same error :(.

[rniebecker]

Ok, so 1.1.1 and 1.1.2 I can't test because of the error in my last post. 1.1.4 is having the same issue as 1.1.7 and 1.1.9.

Will stick with 1.0.7.1 for now.

@Episodex if you have Skype add me, my Skype Id is niewi1, I can sent you the lib then.

Cheers, Ralf

[sannies]

Some versions accidentally used java 8 API not available on android. I never published these I guess I will need a way to reproduce on my phone to be able to debug and understand. Could you perhaps create a HelloWorld-like activity that performs the operation you are doing and reliably fails?

rniebecker notifications@github.com schrieb am Di., 29. Sep. 2015 9:46 PM:

Ok, so 1.1.1 and 1.1.2 I can't test because of the error in my last post. 1.1.4 is having the same issue as 1.1.7 and 1.1.9.

Will stick with 1.0.7.1 for now.

@Episodex https://github.com/Episodex if you have Skype add me, my Skype Id is niewi1, I can sent you the lib then.

Cheers, Ralf

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-144168451.

[Episodex]

I can confirm that it works a lot better with 1.0.7.1. I mean I managed to append tracks totaling over 10 minutes with sound. It still crashes on one of my test sets of videos but I will investigate it later.

Tomorrow I will prepare simplest possible working example with merging two videos where one is over 100 seconds long. Hopefully this will reproduce the problem.

[sannies]

Thank you. I would really like to make it work.

Episodex notifications@github.com schrieb am Di., 29. Sep. 2015 10:39 PM:

I can confirm that it works a lot better with 1.0.7.1. I mean I managed to append tracks totaling over 10 minutes with sound. It still crashes on one of my test sets of videos but I will investigate it later.

Tomorrow I will prepare simplest possible working example with merging two videos where one is over 100 seconds long. Hopefully this will reproduce the problem.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-144185054.

[Episodex]

Hi, I created and uploaded test project to GIT: https://github.com/Episodex/TestIsoparser

You need to manually change videos list in the code. I left mine which are not meaningful to you at all, but you can see how many I try to merge. If you can't reproduce this with your videos then I will shoot and upload some mine.

Let me know if something is missing in this repo. That was first time I published something to GIT and I'm counting that Android Studio did it job well here ;).

Feel free to post pull requests there if I do something wrong using your library. This will help me a lot!

[rniebecker]

Hey Sebastian,

is this fixed now? And if yes, where/when can I get the new version? ;)

Liebe Grüße nach Hamburg, Ralf

[sannies]

checkin closed the issue before the release was done. The staging repository is here: https://oss.sonatype.org/content/repositories/comgooglecodemp4parser-1080 I'll push it to central if you confirm that it's working.

[Episodex]

First quick test showed that it didn't help in my case :(. Still NoSuchElementException. I'll try to dig deeper tomorrow also using my small test project.

[sannies]

Ohh my ... I removed even more clutter and added logging in version 1.1.11 - it might help but I'm just guessing. Repository: https://oss.sonatype.org/content/repositories/comgooglecodemp4parser-1081

On my Galaxy Note 3 I can no longer reproduce the issue with >=1.1.10 If you'd give 1.1.11 a try and check what the log says (I added logging in the DefaultSampleList in case an IOException is thrown )?

[Episodex]

With 1.1.11 I have an error still.

Stacktrace:

DefaultMp4SampleList:java.io.IOException: mmap failed: EPERM (Operation not permitted)
        at java.nio.MemoryBlock.mmap(MemoryBlock.java:125)
        at java.nio.FileChannelImpl.map(FileChannelImpl.java:257)
        at com.googlecode.mp4parser.FileDataSourceImpl.map(FileDataSourceImpl.java:58)
        at com.googlecode.mp4parser.BasicContainer.getByteBuffer(BasicContainer.java:223)
        at com.googlecode.mp4parser.authoring.samples.DefaultMp4SampleList.get(DefaultMp4SampleList.java:176)
        at com.googlecode.mp4parser.authoring.samples.DefaultMp4SampleList.get(DefaultMp4SampleList.java:1)
        at com.coremedia.iso.boxes.mdat.SampleList.get(SampleList.java:41)
        at com.coremedia.iso.boxes.mdat.SampleList.get(SampleList.java:1)
        at java.util.AbstractList$SimpleListIterator.next(AbstractList.java:55)
        at java.util.AbstractList$SubAbstractList$SubAbstractListIterator.next(AbstractList.java:201)
        at java.util.AbstractCollection.toArrayList(AbstractCollection.java:349)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:339)
        at java.util.ArrayList.addAll(ArrayList.java:188)
        at com.googlecode.mp4parser.authoring.tracks.AppendTrack.<init>(AppendTrack.java:74)
[...]
Caused by: android.system.ErrnoException: mmap failed: EPERM (Operation not permitted)
        at libcore.io.Posix.mmap(Native Method)
        at libcore.io.ForwardingOs.mmap(ForwardingOs.java:111)
        at java.nio.MemoryBlock.mmap(MemoryBlock.java:122)
        at java.nio.FileChannelImpl.map(FileChannelImpl.java:257)
        at com.googlecode.mp4parser.FileDataSourceImpl.map(FileDataSourceImpl.java:58)
        at com.googlecode.mp4parser.BasicContainer.getByteBuffer(BasicContainer.java:223)
        at  com.googlecode.mp4parser.authoring.samples.DefaultMp4SampleList.get(DefaultMp4SampleList.java:176)
        at   com.googlecode.mp4parser.authoring.samples.DefaultMp4SampleList.get(DefaultMp4SampleList.java:1)
        at com.coremedia.iso.boxes.mdat.SampleList.get(SampleList.java:41)
        at com.coremedia.iso.boxes.mdat.SampleList.get(SampleList.java:1)
        at java.util.AbstractList$SimpleListIterator.next(AbstractList.java:55)
        at java.util.AbstractList$SubAbstractList$SubAbstractListIterator.next(AbstractList.java:201)
        at java.util.AbstractCollection.toArrayList(AbstractCollection.java:349)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:339)
        at java.util.ArrayList.addAll(ArrayList.java:188)
        at com.googlecode.mp4parser.authoring.tracks.AppendTrack.<init>(AppendTrack.java:74)

I can test more in the evening using my small app.

[Episodex]

Hi, I just started debugging again (haven't got time before) and I realized that this error is now in different place. Previously it was thrown during building of the movie, and all tracks before this were appended ok. But now it's thrown during appending tracks. The code never reaches building phase. Maybe this will help you? I'll continue with my tests.

[Episodex]

OK, I tested it more with my small test app. I looks like the error is not related to the length of single clip, but to the length of total resulting video. In my test app I get the error during build of video. I also have a Note 3 and for me the border length of video seems to be around 9 minutes. I was testing with clips longer than a minute and around 30 seconds long. I used same clips again and again to build up video long enough. It was ok, till I exceeded around 9 minutes.

Please test with a video of total length of let's say 11 minutes (you can just reuse same clip 20 times).

[sannies]

Ok, I can reproduce as well and I think I understood what's going on. Dalvik is a 32bit VM even if it runs on a 64Bit Android OS. I'd therefore conclude that the addressable memory per app is 2GB. Even though I don't use the heap the dalvik VM nevertheless needs 'address space' to map parts of the source files (chunks) into the memory with the FileChannel#map operation. In this case we don't run out of memory in the strict sense - I'd say we run out of address space!

My first patch reduced the amount of memory map operations and prevented certain parts to be mapped into memory twice thereby saving address space and that pushed the point of failure simply to a later point. At that point I still thought that it would have something to do with mapping files into memory twice but it had nothing to do with that.

I think that a simple DataSource implementation that reads the file to a ByteBuffer on the heap on a call to #map will make everything work. I'll check that tomorrow.

Best Regards, Sebastian

2015-10-04 14:08 GMT+02:00 Episodex notifications@github.com:

OK, I tested it more with my small test app. I looks like the error is not related to the length of single clip, but to the length of total resulting video. In my test app I get the error during build of video. I also have a Note 3 and for me the border length of video seems to be around 9 minutes. I was testing with clips longer than a minute and around 30 seconds long. I used same clips again and again to build up video long enough. It was ok, till I exceeded around 9 minutes.

Please test with a video of total length of let's say 11 minutes (you can just reuse same clip 20 times).

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-145340120.

[Episodex]

Thank you for looking into this! I'm looking forward to results of your change :).

[HannahMitt]

Thanks for checking this out. Wanted to echo that I'm seeing the same issue running version 1.1.9 of the lib on Android 5.1.1 (Nexus 5). I get: "Throwing OutOfMemoryError "pthread_create (1040KB stack) failed: Try again"" shortly before a java.util.NoSuchElementException which fails on the line: new DefaultMp4Builder().build(result); when I'm appending videos.

Videos that are only a few seconds long work fine. I'm only appending 2 1-minute videos, but the rest of my app already uses a fair bit of memory. Might account for why I fail earlier.

[sannies]

Quick update. Didn't completely work cause after writing a sample there's still a strong reference to it even though there should only be a soft reference. Ergo I have to check which object holds it. I'll find the time tomorrow. Best Regards, Sebastian

Hannah Mittelstaedt notifications@github.com schrieb am Di., 6. Okt. 2015 9:31 PM:

Thanks for checking this out. Wanted to echo that I'm seeing the same issue running version 1.1.9 of the lib on Android 5.1.1 (Nexus 5). I get: "Throwing OutOfMemoryError "pthread_create (1040KB stack) failed: Try again"" shortly before a java.util.NoSuchElementException which fails on the line: new DefaultMp4Builder().build(result); when I'm appending videos.

Videos that are only a few seconds long work fine. I'm only appending 2 1-minute videos, but the rest of my app already uses a fair bit of memory. Might account for why I fail earlier.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-145973202.

[sannies]

That took a while. 1.1.13 is here: https://oss.sonatype.org/content/repositories/comgooglecodemp4parser-1082

Please use com.googlecode.mp4parser.FileDataSourceViaHeapImpl for instead of FileDataSourceImpl.

            DataSource dataSource = new FileDataSourceViaHeapImpl(file);
            Movie part = MovieCreator.build(dataSource);

The chunks (group of samples) are loaded on the heap instead 'memorymapped'. The references to the chunks are SoftReference and can be gc'ed if memory is low. Same is true for the memory mapped stuff but unfortunately no one can guarantee that the memory is really freed: See http://bugs.java.com/view_bug.do?bug_id=4724038 I hope it works now for all of you.

[Episodex]

I had rather bad day today. Just until this moment! You saved my day :). Now it works for my big videos! Thank you very much for your effort on fixing this! I will have more time for testing this over weekend but it looks very promising now. Please let us know when 1.1.13 is published and I can switch back to using gradle for isoparser.

[sannies]

I'll publish as soon as it's working. You might add this to your gradle scripts to use the staging repository:

repositories { maven { url "https://oss.sonatype.org/content/repositories/orgmp4parser-1001/" } }

2015-10-07 22:03 GMT+02:00 Episodex notifications@github.com:

I had rather bad day today. Just until this moment! You saved my day :). Now it works for my big videos! Thank you very much for your effort on fixing this! I will have more time for testing this over weekend but it looks very promising now. Please let us know when 1.1.13 is published and I can switch back to using gradle for isoparser.

— Reply to this email directly or view it on GitHub https://github.com/sannies/mp4parser/issues/109#issuecomment-146312680.

[sannies]

If I don't hear anything it's typically a good sign. Just send me an email if you need a release pushed to central. The staging repository won't be there for long as sonatypes deletes them after some time