Self-hosting with no account

[Victor239]

Is your feature request related to a problem? Please describe. I prefer to self-host my data for privacy reasons.

Describe the solution you'd like Having access to a docker-compose.yml which for most users requires no customisation to launch a self-hosted instance.

Describe alternatives you've considered Calibre-web is what I currently use.

Additional context N/A

[DavidLazarescu]

Thank you for the suggestion. We will be working on a docker image very soon. We don't have experience with this technology though so it might take a bit. Any contributions from people with docker experience would be very much appreciated.

[gothrek22]

@DavidLazarescu I did a small PR with just the basic docker image that builds, though I'm not really a user of Librum, so can't really check if it works as intended.

34 if you need any support with that, do let me know.

[petruut]

would love to see this as well - I'd love to run this on my home lab!!

[DavidLazarescu]

would love to see this as well - I'd love to run this on my home lab!!

We are currently working on it. Should be done quite soon.

[VGerris]

you can p.m. me if any help is needed, I have experience with docker. too.

[DavidLazarescu]

Hey @VGerris, that would be very much appreciated. Where can I PM you?

[VGerris]

try my handle at gmail :). you can also share the PR and then we go from there - explain what you want and need etc.

[DavidLazarescu]

try my handle at gmail :). you can also share the PR and then we go from there - explain what you want and need etc.

I have already merged some PRs from a contributor (from commit 68c202c75db7a2026130a4f216ee306b5ca07d77 on wards) who was working on the docker image for the server (https://github.com/Librum-Reader/Librum-Server), but they never managed to continue. I suppose they got stuck at the MSSQL dependency while creating a docker image for the server.

Is that something you think that you could setup? I don't have any experience with Docker so I would not yet know how to set that up myself, but I'd be happy to help with any questions regarding the services of the server etc.

[VGerris]

hi, aha I see now, there is a Dockerfile and a script. The Dockerfile only seems to build the server. First of all, I highly recommend to look at something like : https://www.freecodecamp.org/news/the-docker-handbook/ . There was a time I did not know docker, as a developer, it's really essential to know. It basically is technology based on Linux cgroups and namespaces among others. The Dockerfile basically represents the building steps of an image that can be run.

I will take a look at what there is currently and see what is a good step forward. I see the script has dev in the name so I'll focus on that. Note that if you want people to run it in Kubernetes for example, the approach may be different ( then again, I guess a bookreader may not need to scale globally :) ).

The way forward by my knowledge is either:

• create the mysql database in the same docker image and keep everything in one container ( simplest )
• use docker compose to create two containers : 1 with the server app, 1 with MySQL ( extra dependency )

• script another docker container and set it up with shell script

  The person opening this ticket referred to ; https://github.com/linuxserver/docker-calibre-web - there are some examples there to look at. I'll give it a shot and make a PR when done, it will take some time. Not sure what you had in mind yourself to put up, or was that the stuff currently there?

[VGerris]

you mention here that in the appsettings json db and blob storage need to be setup : https://github.com/Librum-Reader/Librum-Server . However , I do not see an example. Can you provide those for a local mysql database and whatever is needed for the blobs ? The way to make it work in docker would be to simply add the file which such config, install mysql and it should work.

[DavidLazarescu]

Hey, thanks for giving it a try. The docs are a bit outdated since we switched some technology. From what I know its not possible to set up blob storage containers locally.

[VGerris]

hi,

You're welcome. I got a little further but I think you need to provide more documentation on how to continue, because it seems to tied to Azure ( which I think by itself is not a good idea for many reasons, but I'll not go into that ). One thing to improve is that the current container does not set any variables to omit for example what you check in Program.cs line 19, namely if it is self-hosted, so the first crash is at that. To fix it one way, you can change the docker command : docker run --env LIBRUM_SELFHOSTED=true -it --rm -v $(pwd):/librum --entrypoint "bash" -p 7084:7084 -p 7412:7412 librum The next error is : root@2778d0fe8a46:/app# dotnet Presentation.dll Running in selfhosted mode, skipping AzureKeyVault configuration Unhandled exception. System.IO.InvalidDataException: Failed getting the JWT Key at Presentation.DependencyInjection.ConfigureJwt(IServiceCollection services, IConfiguration configuration) in /app/src/Presentation/DependencyInjection.cs:line 110 at Program.

$(String[] args) in /app/src/Presentation/Program.cs:line 35 at Program.

(String[] args) Aborted (core dumped) root@2778d0fe8a46:/app#

You can go from there when it comes to fixing it I suppose. To get there, just run the script with the above modification.

I have no time to fix the app and do not have much dotnet experience ( or love :) ) but if you are able to make it work locally I can help you with the docker part, although I do suggest to give it go yourself, that's the best way to learn.

From what I can see, your connection setup happens partly in DependencyInjection.cs along with the Blob storage. Is there a specific reason to use such vendor bound technology? There is an emulator, which is Windows only which defiess the whole point of platform independence ( and I am certainly not using Windows for any development ).

I suggest to pick a storage that is similar that is easy to setup in both cloud and local on all major operating systems. Given that this app is likely just going to store pdf / epub and such, why not use S3 storage? You could also look at : https://github.com/qoollo/bob One way or the other the dev env should be able to have it working, ideally without cloud. In the end it all depends on what you want to achieve.

I hope this helps, I could not easily hack around in the source to try some things and without that as a really short feedback loop, I do not have time to fix anything code wise. Let me know how it goes and if help is needed with docker, good luck!

[pslobo]

I came here exactly looking for a self-hosted option. Happy to see that it's being worked on. Do let me know if there's anything I can help with. In regard to blob storage, if you're using S3 compatible API calls, then something like Minio would easily fit the bill for self-hosted solutions, with the benefit that it can also be deployed to Kubernetes and just about any cloud provider

[tralph3]

In case you need a quick overview of how to work with Docker, I explained it in a PR I made to a music app because the dev didn't know Docker either. The explanation is referencing the Dockerfile I included in the PR of course, but should be good enough to somewhat get up to speed, maybe. Do let me know if it's useful to you:

https://github.com/swing-opensource/swingmusic/pull/116

[DavidLazarescu]

@tralph3 thank you, I have been reading about docker in the last days and your comment definitely helped me to connect some concepts!

The main problem I see is mocking the dependencies like the MSQL database, the keyvaults and the blob storage to all work locally. There is currently someone working on this who is trying deal with this, so I can not give a very accurate estimation of how long this will still take. I hope that the docker image will be done soon though.

[VGerris]

Hi,

If you need help, just write concretely what you're stuck with. It shouldn't be too hard just running things in docker, easiest is with compose.

MySQL is just a matter of creating a container based on the MySQL image and set up the connection, there should be many examples online for for example a LAMP stack.

The only challenge I saw was the blob storage : your code depends on it, so it needs to be modified for dev in the way you want, I suggested options earlier.

No rush, take time to learn, anyone doing so will benefit :).

Good luck!

On Fri, Sep 22, 2023, 07:46 David Lazarescu @.***> wrote:

@tralph3 https://github.com/tralph3 thank you, I have been reading about docker in the last days and your comment definitely helped me to connect some concepts!

The main problem I see is mocking the dependencies like the MSQL database, the keyvaults and the blob storage to all work locally. There is currently someone working on this who is trying deal with this, so I can not give a very accurate estimation of how long this will still take. I hope that the docker image will be done soon though.

— Reply to this email directly, view it on GitHub https://github.com/Librum-Reader/Librum/issues/28#issuecomment-1730835241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGB3CMFYCBHX7TFQKRPLUDX3UQ4VANCNFSM6AAAAAA4LITP7E . You are receiving this because you were mentioned.Message ID: @.***>

[DavidLazarescu]

hi,

You're welcome. I got a little further but I think you need to provide more documentation on how to continue, because it seems to tied to Azure ( which I think by itself is not a good idea for many reasons, but I'll not go into that ). One thing to improve is that the current container does not set any variables to omit for example what you check in Program.cs line 19, namely if it is self-hosted, so the first crash is at that. To fix it one way, you can change the docker command : docker run --env LIBRUM_SELFHOSTED=true -it --rm -v $(pwd):/librum --entrypoint "bash" -p 7084:7084 -p 7412:7412 librum The next error is : root@2778d0fe8a46:/app# dotnet Presentation.dll Running in selfhosted mode, skipping AzureKeyVault configuration Unhandled exception. System.IO.InvalidDataException: Failed getting the JWT Key at Presentation.DependencyInjection.ConfigureJwt(IServiceCollection services, IConfiguration configuration) in /app/src/Presentation/DependencyInjection.cs:line 110 at Program. $(String[] args) in /app/src/Presentation/Program.cs:line 35 at Program.(String[] args) Aborted (core dumped) root@2778d0fe8a46:/app#

You can go from there when it comes to fixing it I suppose. To get there, just run the script with the above modification.

I have no time to fix the app and do not have much dotnet experience ( or love :) ) but if you are able to make it work locally I can help you with the docker part, although I do suggest to give it go yourself, that's the best way to learn.

From what I can see, your connection setup happens partly in DependencyInjection.cs along with the Blob storage. Is there a specific reason to use such vendor bound technology? There is an emulator, which is Windows only which defiess the whole point of platform independence ( and I am certainly not using Windows for any development ).

I suggest to pick a storage that is similar that is easy to setup in both cloud and local on all major operating systems. Given that this app is likely just going to store pdf / epub and such, why not use S3 storage? You could also look at : https://github.com/qoollo/bob One way or the other the dev env should be able to have it working, ideally without cloud. In the end it all depends on what you want to achieve.

I hope this helps, I could not easily hack around in the source to try some things and without that as a really short feedback loop, I do not have time to fix anything code wise. Let me know how it goes and if help is needed with docker, good luck!

Sorry, I have just seen your reply.

"Is there a specific reason to use such vendor bound technology?" What exactly do you mean with that?

"Given that this app is likely just going to store pdf / epub and such, why not use S3 storage?" Because I am using Azure for all of my storage services and I do not plan to switch Librum's complete cloud infrastructure to a platform that I am not familiar with and which does not work well with asp.net, just for the docker image.

[DavidLazarescu]

If you need help, just write concretely what you're stuck with.

Another contributor is working on that part, so I can't really answer that. It seems like they have managed to set up the MSSQL server and blob storage but are still working on finishing the image.

[VGerris]

hi,

With vendor bound, I mean using technology from a vendor that then locks you in. There are platform independent technologies. One reason to use those is that if Microsoft doubles the price tomorrow you can just redeploy your stuff elsewhere or on a local computer. S3 storage also is available on Azure cloud and there are libraries for C#. In general, do not get locked in into any vendors ecosystem, that is after all what most try and especially Microsoft has a habit of it. If you look at Google cloud you will see that pretty much everything is open standards based and as such easy to move away from if wanted. The question regarding storage was also about the storage type. Block storage is typically used for large amounts of data to be quickly searched - S3 is more generic so called object storage and pretty much every cloud has it. Meaning, if you do not need the properties of blob storage it's probably much better to use S3 storage. In that case, if you would move something, say storage, your only code changes would be the URL and possible the URL type.

I hope that clarifies things. And if you like a European cloud, check out Binero, it's openstack based and has S3 storage and it's running in Sweden ( affiliation :) ). Sounds like the docker thing will be figured out,again I highly recommend trying things yourself if you develop, it's great technology. Give Linux a try on the desktop and you can see how well it integrates ( shared memory ), there is VScode on Linux, I use it almost daily. good luck!

[DavidLazarescu]

Thanks, I didn't know S3 was available on Azure as well, I thought it was a AWS thing, I'll definitely read more about it.

I am planning to learn more about Docker but that's a bit of a challenge time-wise, so I am very happy that another contributor takes this over for now and I can concentrate on the app and similar.

Also, I'm not sure what exactly you mean with "Give Linux a try on the desktop and you can see how well it integrates". If you mean that I should give Linux a try, I am using it daily for multiple years already :)

Thank you for your comments again!

[VGerris]

hi, I suspect, because I have mostly bad experience with Azure, but Google Cloud, Amazon, OpenStack ( Binero ) and I think others have it, but otherwise it is really easy to deploy and free with : https://min.io/

Aha, great you use Linux, you sounded like an all-Microsoft user :). With the integration I mean the memory, docker on Linux shares memory with the OS directly, on MacOS or Windows there is always some form of virtual machine in between that has it's own memory.

Your welcome, good luck once again, gonna try your product out when it's running locally :).

[magick93]

In general, do not get locked in into any vendors ecosystem, that is after all what most try and especially Microsoft has a habit of it

Completely agree.

Its a shame to see this project being so dependent on M$ technologies, especially mssql.

[DavidLazarescu]

The client does not depend on MS technologies, the server does, since that is the backend technology that I know best.

I’ll happily take PRs if any of you have recommendations or want to decouple it, but I am not familiar with AWS or others and don’t have the time to learn that just to not depend on microsoft products.

On Tue, Sep 26, 2023 at 06:33, magick93 @.***(mailto:On Tue, Sep 26, 2023 at 06:33, magick93 < wrote:

In general, do not get locked in into any vendors ecosystem, that is after all what most try and especially Microsoft has a habit of it

Completely agree.

Its a shame to see this project being so dependent on M$ technologies, especially mssql.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[3036662]

Hi, thank you for your hard work. I made a server version that does not use azure, uses a local mariadb database, and a local file storage. It works under linux. If anyone is interested I can share the information.

[DavidLazarescu]

Hi, thank you for your hard work.

I made a server version that does not use azure, uses a local mariadb database, and a local file storage. It works under linux. If anyone is interested I can share the information.

Hey, sorry for the delay. The contributor working on the Docker image stopped and I was really busy fixing bugs and similar on Librum.

It would be great if you can share your work here. I could link it as unofficial self-hosted solution if it works fine.

[3036662]

In fact, there are only a few small changes in the server code - a module for working with mariadb has been added, and some settings have been changed so that the database structure is created when the server is first started. The files are stored on the server's disk space. I also allowed myself to add the ability to select a server in the client application using a .conf file. I can say beforehand that the application works fine on the client version 0.9.2, tomorrow I will check on the new release. Thanks for the bugs fixed, I really appreciate your work.

[DavidLazarescu]

That sounds great! Feel free to make a PR so that we can merge the config file that you added to the official repo. I can also add a setting for that to Librum so that it's possible to switch to a self hosted server easily during runtime.

[3036662]

In continuation of our conversation. I made a draft of forked version of server, you can take a look at https://github.com/3036662/Librum-Server/tree/gitHubFork_selfHosted I've also made PR for client, to enable server url setting.

[DavidLazarescu]

Awesome, I took a quick look and it looks great so far. I am pretty busy right now so I'll need some time to look over and test the PR you made to the client repo.

I hope that we can get this into the next release of Librum.

[3036662]

I think the good Idea is not to fork the server, but continue the thought about "LIBRUM_SELFHOSTED" variable, so admin can switch technology from local to azure and backward when configure server. I will work on it, hope you will consider including changes in your repo.

[DavidLazarescu]

That is a good idea, Id be happy to merge it if it works well

[3036662]

I've made a PR for server. You can take a look when you have time. It will allow to configure and run in self-hosted mode with MySql or MariaDB

[DavidLazarescu]

We have now added a docker file and the ability to run Librum's Server as a service on linux.

I am therefor closing this issue.

[manan-gup]

Does the docker backend work with the flatpak or just the tar.gz? And is there any setup needed to make the frontend work with it?

[DavidLazarescu]

The docker backend works with every distribution of Librum, thus also the flatpak. There is a guide in the docs of Librum-Seever on how to set it up. You just need to make sure to be in the flatpak relative root folder if you want to find the config files of Librum when installed via flatpak. Afaik that's ~/.var and then flatpak or apps and then com.librumreader.librum

[manan-gup]

Thanks, I'll give it a go and report back if there are any issues.

[manan-gup]

I built the docker image and the used the compose file to run both services. Then made the config changes mentioned here. But when I start librum flatpak I still get the login form and if I try to make an account I get an error. Am I doing anything wrong?

[manan-gup]

Ok I had put the port as 5001 instead of 5000 in the config file. I could make an account and login after changing that. Is there any way to confirm from within the client that I'm connected to the local server?

[DavidLazarescu]

It is connecting to what ever domain is in the Librum.conf file. If you haven't provided an OpenAI key, you should see that going to a book, selecting some text, clicking on > and then "Explain" will give you no output.

[manan-gup]

Ah ok, thank you so much.

[ilkersigirci]

Thanks for this. I have successfully connect the client app with the self-hosted server. The only problem is that the server doesn't store the actual file (pdf, epub). Will there be any future work around this? This way, one can bind existing books to docker volume (var/lib/librum-server/librum_storage) and can read the already downloaded books.

[DavidLazarescu]

Thanks for this. I have successfully connect the client app with the self-hosted server. The only problem is that the server doesn't store the actual file (pdf, epub). Will there be any future work around this? This way, one can bind existing books to docker volume (var/lib/librum-server/librum_storage) and can read the already downloaded books.

What exactly do you mean with "the server doesn't store the actual file (pdf, epub)". The server does store all files in its own folder.

[ilkersigirci]

For example, I added a book called "1984.pdf" via client application. In the server it stored as "5a6f8d9d-6f91-4d6d-b201-aa70a19ccf35", not as pdf file. If it can store pdf files, I could directly import my existing book library by just binding the library as docker volume.

[DavidLazarescu]

Ah, I understand. It does store them as pdf files, it just renames them according to their uuid and removes the pdf extension.

[ilkersigirci]

Hmm, I get it. Will you consider changing this behavior and store the files as is?

[DavidLazarescu]

We can't store files by their original names because they are not unique. Also changing this behavior on the server would not be possible because we are already storing tens of thousands of files this way and it would cause them to malfunction.

[ilkersigirci]

I see. I think my use-case can't be applied to Librum. Thanks for the fast response

[DavidLazarescu]

Wouldn't it be possible for you to simply import your files into Librum?

[ilkersigirci]

Yes it is possible but as a self-hosted enthusiast, I prefer read-only archive for my files and expect the application to figure the rest. Just like jellyfin did for movies/tv shows and immich did for photos.

[Victor239]

We can't store files by their original names because they are not unique.

Can't it be done like how most OSes do by simply looking at the file type + file size? Only if they match does it then prompt you to rename a file e.g. adding a (1) suffix.