search

OpenNewsLabs / autoEdit_2

Fast text based video editing, node Electron Os X desktop app, with Backbone front end.
https://opennewslabs.github.io/autoEdit_2/
MIT License
421 stars 56 forks source link

Linux version #36

Closed probonopd closed 6 years ago

probonopd commented 7 years ago

Is there any reason not to publish a Linux version?

pietrop commented 7 years ago

Hi @probonopd,

I am looking into both a windows and a Linux version.

In order to do so there as need to :

  1. Add ffmpeg and ffprobe binaries specific to these systems.which I have done using the static ffmpeg and static ffprobe but needs testing.

  2. Also it needs going through the code base and changing any accourance of / in file paths across the project to the node path library to make it cross platform compatible.

  3. It needs compiling into Linux and testing on a linux machine (or virtual machine) to see what else if anything would need adjusting, by looking at the errors in the console if any.

  4. I don't have a windows or a Linux machine, so I am setting up virtual box, but it's not as fast of a workflow.

I'd be curious to learn more about your use case tho, what would you be using autoEdit for?

probonopd commented 7 years ago

Hi @pietrop I don't have a use case yet, just wanted to try it out. Instead of setting up your own Linux build machine, you could also use a service like Travis CI. I use it a lot for my work on AppImages. Or, you could use the Open Build Service.

pietrop commented 7 years ago

Ah, that sounds interesting, will need to look into that when I circle back to this.

For linux which distribution would work best for you?

sedubois commented 7 years ago

+1 for a Windows version! I have macOS but collaborate with a colleague who uses Windows.

pietrop commented 7 years ago

@sedubois thanks for this input, quick question to you both, would a web app be just as good or are there preferences on it running as a desktop app?

sedubois commented 7 years ago

I think a web app would be more flexible, wouldn't it? No need for cumbersome installations etc. The question is more on your side, how do you keep track of heavy files etc.

sedubois commented 7 years ago

An API + web app would also make it easier for people to collaborate together.

pietrop commented 7 years ago

That's a good point, the reason why I started with a desktop app is that if you are working with files from a camcorder, in this setup you could add them to autoEdit straight from the card (using ffmpeg to make audio and video preview and ffprobe to read metadata info for the EDL)

A web app has the trade of that it could be mobile responsive (if you check the demo on the phone, the front end already is) and work across devices. But it might need to have limits on file size that can be uploaded.

Altho for example it just blew my mind seeing that google drive can let you upload up to 5TB in size, I wonder how long that would take to upload, and what are the feasibility of that scenario.

Ideally you'd have both desktop and web app, and they sync to the same account. So that you can choose what to use based on the circumstances. In this scenario, the original media, often huge in size does not sync, just the metadata to reconnect the EDL and the audio and video previews("proxies") if that makes sense.

Something to think about tho, is that the web app altho has costs for keeping it up and running, especially if it needs a backend, and is therefore harder to keep it freely available.

sedubois commented 7 years ago

@pietrop I have the feeling that going for several types of clients would bring more overhead, better to concentrate on one access point which is useful to as many people as possible. So the web app looks like the better way, IMHO. For sure as you mentioned it's possible to manage even very big files in this way, like the web client for Google Drive shows. I guess the files just need to be downloaded to the local machine before being manipulated, and re-uploaded as a new version once your editing is done. Maybe something like GitHub LFS could allow to transfer only the file diffs.

Also as this is open-source software, I think that hosting costs is one of the things that you needn't worry about. With an OSS project it's normally possible to do everything for free. E.g the frontend or microservices could be dynamically deployed with Zeit Now or frontend could also be statically deployed for free with Netlify. I also like GraphQL and Graphcool hosts open-source projects for free. And there's also plenty of free stuff for testing. Etc.

sqr commented 6 years ago

I must say a Linux or Windows version would be super useful to me, I work in a newsroom where people use both OS X and Windows. I personally don't own any OS X machines so unfortunately I can't try the software, it seems to be very promising!

jsdephillips commented 6 years ago

Hi @pietrop I am also interested in a version I can run on PC. I'm working on a documentary and came across Auto Edit. It seems to be a perfect tool for the type of project I am working on in AVID Media Composer. I have hours of interviews that I am using to build the narrative of a documentary. We have tried Auto Edit on a Mac and it works wonderfully. Unfortunately, I have PC in my edit suite. Web based has some drawbacks for me as some of the interview files I have are upward of 12 gigs. So upload time would be an issue. Please let me know when you have a PC version I can try out. Thank you so much. Jacquelyn

sedubois commented 6 years ago

Maybe it could be one JS codebase delivered on desktop with Electron.

pietrop commented 6 years ago

Hi @jsdephillips, Thanks for your message, glad you found it helpful, "digital paper-editing of docs" is the main use case I built autoEdit for. On that note you might find this interesting, some notes on the underlying analogue workflow.

As mentioned in previous comment these are the things that would be needed to make the Linux and Windows versions. I've done the first one, but could use some help with the rest of the list.

I'd be happy to guide anyone who wants to have a go at this, as I might not have the time to dedicate to it at the moment.

I also don't have a Windows or a Linux machine, so I'd welcome help in figure out an effective workflow to work around this.

@sedubois @probonopd let me know if you have any thoughts, or would like to get involved.

probonopd commented 6 years ago

It needs compiling for Linux and testing on a linux machine (or virtual machine)

Started working on it at https://github.com/probonopd/autoEdit_2/blob/patch-1/.travis.yml. It is being compiled on Travis CI which means that it can be built without the need to set op Linux on a local machine.

But getting

0.67s$ npm run test
> autoEdit2@1.0.8 test /home/travis/build/probonopd/autoEdit_2
> jasmine
module.js:472
    throw err;
    ^
Error: Cannot find module '../../interactive_transcription_generator/video_metadata_reader/index.js'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/travis/build/probonopd/autoEdit_2/spec/video_metadata_reader/video_metadata_reader_spec.js:2:22)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)

https://travis-ci.org/probonopd/autoEdit_2/builds/363017523#L1428-L1447

Do you have an idea? For now I am just commenting out the test entirely.

probonopd commented 6 years ago

Change path references across the code base, in other words changing any instance of / in file paths across the project to the node path library to make it cross platform compatible.

This sounds like something you could do best since you know your codebase best...

probonopd commented 6 years ago

Hey, it's a start... running the AppImage on Xubuntu 16.04.

Is getting this Bluemix thing free? Oops, https://www.ibm.com/watson/services/speech-to-text/ says

To get started, you will create a Lite Plan (no charge) instance of the Speech to Text service, which is capped at 100 free minutes of input audio. Your Lite Plan instance will be deleted after 30 days of inactivity

That's a bummer. I hope there will be alternative (local) speech recognition engines supported. Since I will not sign up for the IBM service, I'd be happy if someone else could test the AppImage from https://github.com/probonopd/autoEdit_2/releases/ and report back here.

pietrop commented 6 years ago

Thanks @probonopd !

Great News about the linux version!

re- IBM Bluemix Fair enough, let me see if I can find a linux user, I have asked a friend who as Linux machine if he can test it out, as well as given a shout out on twitter. But perhaps @sqr might be interested in trying out?

re-tests I am going to have a look as soon as I have a second, to try and get some clarity on what is causing that, but disabling the tests seems fine for now.

re-changing the path Same as above, I can do a quick pass through the code as soon as I get a sec, and then test the app out before commit to master. Altho if anybody else wants to make a pull request. it's a matter of going through lib folder.

Really excited seeing this coming together on other platforms!

pietrop commented 6 years ago

re-linux version It seems like it works! Curiosity took over and I downloaded a virtual box and ubuntu

Keyboard shortcuts also work, but need to use ctrl instead of cmd.

ubuntu

@probonopd I'd love to document how to set it up with Travis CLI and connect that to the autoEdit repo to automate the process. What's the best way to go about that?

For now I tried building the Linux version on OSX, after looking at your fork and changing the package.json to be

    "linux": {
      "category": "",
      "packageCategory": "GNOME;GTK;VideoEditing;PaperEditing;Transcriptions",
      "description": "autoEdit2 Desktop Client for Linux",
      "target": [
        "deb",
        "zip",
        "AppImage",
        "rpm"
      ],

to be 

 "linux": {
      "category": "",
      "packageCategory": "GNOME;GTK;VideoEditing;PaperEditing;Transcriptions",
      "description": "autoEdit2 Desktop Client for Linux",
      "target": [
        "AppImage"
      ],

I then run 

 npm run build:linux

See Linux release here

https://github.com/OpenNewsLabs/autoEdit_2/releases/tag/1.0.8

re-test It's the tests in metadata reader that are causing the problem, I had hardcoded the path to a local video file. But I've seen that most of the rest of the test coverage has that problem, so need to figure out a better strategy for testing modules that have media files as inputs, perhaps using fixtures etc.. So I'd live it commented out for now..

re-path found an example of something that needs changing as explained above for this to work on windows

probonopd commented 6 years ago

I can send a pull request for the Travis CI build automation if you like. It will build an AppImage on each git push.

pietrop commented 6 years ago

That'll be great! thanks @probonopd !

I have a few questions tho:

Thanks!

pietrop commented 6 years ago

another question re-AppImage @probonopd

I found here that 

AppImages can be downloaded and run without installation or the need for root rights.

Making it executable
You can make the appImage executable as follows:

chmod a+x exampleName.AppImage
Executing it
You can execute an appImage as follows:

./exampleName.AppImage

My question is, if I run the command to make it executable before publishing the app chmod a+x exampleName.AppImage then the user doesn't need to do that on their end? and if that is the case is it possible to incorporated in the deployment script? So that the user only has to download and then double click on the AppImage to get up and running.

pietrop commented 6 years ago

Opened a new issue to talk about the Windows version specifics https://github.com/OpenNewsLabs/autoEdit_2/issues/44

probonopd commented 6 years ago

The user needs to do this. This is how Linux security works. See https://discourse.appimage.org/t/how-to-make-an-appimage-executable/80

Is there a way for Travis CI to also do the same for the OSX version and the Windows

Yes for macOS, no for Windows (there is CircleCI for that).

Is there an easy way to change the script so that it will build an AppImage on each git push only if it's a new version?

No.

pietrop commented 6 years ago

Ok great, thanks @probonopd !

For now how would I add macOS build to the Travis CI?

Is it as simple as adding the line below to the .travis.ym config on line 35 ?

- bash upload.sh ./dist/autoEdit2-*.dmg autoEdit2

Re-Pull request for the Travis CI build automation, sounds good, feel free to do that.

pietrop commented 6 years ago

@probonopd I tried and add your .travis.yml but got the following issue

https://travis-ci.org/OpenNewsLabs/autoEdit_2/builds/364009530

Something to do with permissions denied when trying to get ffmpeg-static and ffmprobe-static repositories.

Did you had this problem as well when you tested it in your fork?

probonopd commented 6 years ago

Sent PR at https://github.com/OpenNewsLabs/autoEdit_2/pull/45.

Did you had this problem as well when you tested it in your fork?

Yes. See my PR that fixes it.

pietrop commented 6 years ago

Closing this as just integrated PR https://github.com/OpenNewsLabs/autoEdit_2/pull/45

For Windows related question, see separate issue https://github.com/OpenNewsLabs/autoEdit_2/issues/44