Linux Server Support

[jmediamanager]

Reported by adam.keun, Mar 13, 2013

Hi just wondering if there is any plans for having linux support for the JMM server?

Apr 21, 2013 dominik.karadeniz

Yes, that would be great. I am dreaming of integrating it in my server setup, with transmission, xbmc.

Everything works like a charm now and is fully automated, but handling anime is still poor. I hope to change that with jmm. Unfortunately, being a windows application severely limits its use cases and modularity.

May 15, 2013 Craige1Emmott

I can tell you now this probably wont happen any time soon. JMM as a whole is completely built around MS technologies. Anything running on Linux will ether require mono or a complete rewrite. Even the API used for communication is a MS technology. It would take one skilled Linux dev with some serious dedication to make this happen.

May 15, 2013 adam.keun

I had a feeling it was built around MS, tis a shame because it's such a great program and I would have loved to integrate it with my linux machine.

[Cazzar]

Currently no, I have not tried it.

I am running a major test as my main DB had failed due to openssl upgrades on archlinux. I am currently re-generating my database via MySQL + Linux + Mono right now. Once we have this, and @maxpiva 's OAuth setup done, I'll be happy to merge Also, for the models issue @bigretromike git submodule update --recursive should resolve it, as I built the software yesterday.

I might push a couple of updates in a moment.

[Cazzar]

For people curious, I whipped up a quick script to monitor file status via Cacti..

This is my current status. I'm still ~4k files hashing. don't mind the two gaps in the graph, as that is more so poller errors due to my own doing.

[bigretromike]

I did update submodules it more of those 'get => ' 'set =>' things there. as VS tell me its not allowed.

nice-desu Cazzar-san 👍

[bigretromike]

Also better command for submodules would be: git submodule update --init --recursive

[bigretromike]

Those are the issues im talking about:

[da3dsoul]

don't forget --remote

[Cazzar]

Give me 5, I'll push my latest changes.

[Cazzar]

@bigretromike try 3862bbaaeed940513fc603410a84897609a7670a

Also, note:

• Need to work out why the custom Pri.LongPath isn't getting into the build output. Current workaround: Manual copy.

[bigretromike]

same errors as the one on screenshots.. it looks like lang errors so something that interpreter language is missing or code is broken (I would put my bet on the first option)

maybe I need to install something extra to VS ?

I have .net framework 4.6.2 sdk + targeting pack + targeting pack (enu) so it should be fine;

[Cazzar]

Is your working directory at least clean?

[bigretromike]

yes? I just clone it fresh, pull submodules, open project in vs, pick build/rebuild/debug and boom

I will check if I even can build normal shoko. same thing. no matter which branch.

[bigretromike]

[Cazzar]

You're not getting the latest c# 5 constructs as that is valid.

On Mon., 8 May 2017, 5:09 pm BigRetroMike, notifications@github.com wrote:

[image: sssssss] https://cloud.githubusercontent.com/assets/6052025/25793699/f13a5cb0-33cd-11e7-84f6-e9c8bb2f1fc5.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/japanesemediamanager/ShokoServer/issues/52#issuecomment-299789999, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8rpmhksd0cSRYXQpyjEmwB9tGM6wr1ks5r3r-TgaJpZM4CYrXG .

[bigretromike]

? any solution for this ?

Im using VSC 2015, acording to msdn:

features that were added in C# 6. These features are available in Visual Studio 2015

So i should be fine, but im not

Will install KB3165756, lets see if this is one of issue fixed;

Nope Update3 didn't fix this.

[da3dsoul]

Go redownload the .net sdk (NDK)

[bigretromike]

@da3dsoul yeah thats my bet also.

Sadly windows is telling me that I have 4.6.x installed even when i uninstalled it because I couldn't force installer to overwrite installation... Looks like another win10 issue with himself.

I have better chances at home with not windows 10.

[Cazzar]

@bigretromike see what happens when you manually build with MSBuild https://msdn.microsoft.com/en-us/library/dd393574.aspx#Anchor_0

[bigretromike]

@Cazzar thank for info, but I will stop messing with this at work today ;-) I don't have time to resolve windows issue today 👍 probably one of my prevent-ransomware application prevent cleaner from cleaning after .net clean up tool - and end up with mess. Will check at home.

[bigretromike]

Took me some time... but...

It looks like windows 10 prevent you from installing any older .net than you already have as system component (which in win10 anniversary is 4.6.2) that is why I was unable to install another version after uninstalling 4.x from add/remove software which where installed before w10anniv_patch.

also hasher have set tool set for v141 which is vs2017 :-) which is my problem

NDP462 is installed correctly. Still having https://github.com/japanesemediamanager/ShokoServer/issues/52#issuecomment-299789999 the issue.

My question would be, does it compile on VSC2015? because everything look like everybody use 2017 edition.

[da3dsoul]

yes , go ahead and change the toolset back

[bigretromike]

changing toolset didn't help at work; I will check with vs2015 at home.

Identical issue with vs2015 at different computer. Let me upgrade to vs2017.. because of no reason

[bigretromike]

Finally. Hasher was setup for windows10 with some version that I was missing and VS2017 did the job. Sadly... After installing vs2017 i can compile it with it, but not with vs2015... so yeah... Thanks for help, now I can build from source once again.

[Cazzar]

We also need hasher preferably ported to linux as well, right now the C# code on my Xeon E3-1270 averages about 4-13 seconds.

Also, thoughts on multi-threading hashing operations? Being bound to one CPU core could be making hashing a little slower, as so far for 9K files, I am at the ~24 hour mark.

[bigretromike]

hmmm for linux we can fallback to c# hasher which is slow. and look for something better in mean time. for start it should be fine, as it will be more of experimental package. there should be a hell-tone of hashers native for linux, and we could use those .. no need to maintain something like that

[maxpiva]

the C++/x86/x64 version is probably near IO bound on spinning disks on latest CPUs. I don't know if multi thread will work, without over-engineering the C++ library. Anyway, we miss an MD4 (ED2K) x64 assembler version, we could add MD5 x64 if we want, there is code already. But there is no code for MD4 for x64. (its possible to adapt the MD5 x64 version), but it will be a pain in the ass.

[bigretromike]

but if its not working on linux then we are not sure if porting wont end up sacrificing that 'speed' on windows machines - or in the end there would be 2 different hashers platform specific, and im more than sure that there are hashers that are good as this on linux platform already and in 99% there are opensource anyway.

[Cazzar]

We could see if we can p/invoke Linux libcrypto for native hashing.

On Tue., 9 May 2017, 3:24 pm BigRetroMike, notifications@github.com wrote:

but if its not working on linux then we are not sure if porting wont end up sacrificing that 'speed' on windows machines - or in the end there would be 2 different hashers platform specific, and im more than sure that there are hashers that are good as this on linux platform already and in 99% there are opensource anyway.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/japanesemediamanager/ShokoServer/issues/52#issuecomment-300064453, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8rpj3lj_Xy06H8WYRTES3pa6sSnN5Xks5r3_iEgaJpZM4CYrXG .

[Cazzar]

Currently we still have hard coded references to \ in the code base These need to be replaced when referencing paths

[bmfrosty]

I know it's been looked at before, but librhash is out there and supports a plethora of hash formats:

https://github.com/rhash/RHash

On Thu, May 11, 2017 at 8:51 AM, Cayde Dixon notifications@github.com wrote:

Currently we still have hard coded references to \ in the code base These need to be replaced when referencing paths

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/japanesemediamanager/ShokoServer/issues/52#issuecomment-300833075, or mute the thread https://github.com/notifications/unsubscribe-auth/AAB8Dgm-pC3LnhgjmfveE3oELaYCjjhoks5r4y51gaJpZM4CYrXG .

[Cazzar]

@bmfrosty we can look into it, namely the reason for thinking about libcrypto is it's implemented in most systems as part of OpenSSL/LibreSSL

Though on other news @bigretromike you might like this, mono 5 was released, which is now using Roslyn, which means in the long run, we should be able to compile shoko on linux as well.

[Cazzar]

http://www.mono-project.com/docs/about-mono/releases/5.0.0/ for reference.

[Cazzar]

http://blog.wezeku.com/2016/10/09/using-system-data-sqlite-under-linux-and-mono/ also will get SQLite working on mono. Downside being needing to build the interop library.

[bigretromike]

its good for start. later we could just use the lib on system

[bmfrosty]

How's it going?

[da3dsoul]

It's coming along, we've got an alpha docker image out

[ElementalCrisis]

Join our discord, this thread is really only updated when a significant milestone has been reached.

[bigretromike]

@bmfrosty thanks to hard work and some extra work from @Cazzar we are having jmm/shoko in docker on linux :-) alpha or not it works

[bigretromike]

@ElementalCrisis I think that this is done, a lot of work was putted into this, and we got running linux version right ? 👍

[da3dsoul]

I wouldn't say done until we make sure everything works and is accessible. There's still things that are only available in the server UI

[bigretromike]

imo that should be done in other issue as the prime goal which was running it on Linux is done. Even it could goes to WebUI issues as missing gui features. IMO alot of us are happy with milestone that linux supoort was achive. This way we could enourage people to start using linux in testing so there will be more up-2-date issues that we could handle one by one vs one-ass-long-issue-that-never-ends.

[da3dsoul]

It's not just WebUI though. Those things don't have endpoints or Desktop access either, else it would be closed

[ElementalCrisis]

Technically it works, but its not usable when you have to use Desktop or Windows Server to still properly run it.

[bigretromike]

as you said many times webui which is essential part of linux version never was meant to replace Desktop, right ? and using Server (on windows) never gave us same thing as Desktop app did. But ok.

[da3dsoul]

My point is that there are things that Desktop and webui can't do that the server UI can.

[bigretromike]

@ElementalCrisis any chance we could make a proper milestone to close this VERY important for all of us Issue ?

[ElementalCrisis]

As far as I know, work on the Linux version has been halted as @cazzar has been busy with other projects. I'm actually not quite sure what's left either.

I've been meaning to set up a milestone for 4.0 so maybe we can add this to it.

[da3dsoul]

As for right now, this is what needs done:

• More logging/progress report on certain things
• make sure that all settings and buttons that exist in Server's UI are accessible in webui and Desktop
• I may be misremembering, but proper cloud UI for login and whatnot
• migration to and from Linux DB (not needed, but would be nice)

[ElementalCrisis]

The .Net Standard refactor is something else that needs to be done before this issue can be closed.

@hidden4003 added all the settings we have endpoints for in Web UI.

[da3dsoul]

Yep. Right now, I'm worried about Desktop, as it's the one missing some

[bigretromike]

@da3dsoul Desktop v2? because we dont port v1 to linux, right.