search

processone / ejabberd-contrib

Growing and curated ejabberd contributions repository - PR or ask to join !
http://ejabberd.im
248 stars 137 forks source link

Ejabberd mod_http_fileserver and mod_http_upload support for AWS RDS and S3 #305

Closed armor009 closed 1 year ago

armor009 commented 3 years ago

Ejabberd mod_http_fileserver and mod_http_upload support for AWS RDS and S3 will greatly enhance usability of ejabberd.

badlop commented 3 years ago

They don't work? What are the error messages?

armor009 commented 3 years ago

They host data on server. I am planning to use AWS EC2 for running the server. The cost of EBS is high. I am considering using RDS and AWS S3. Can there be support for sending files across http using RDS (Postgres) and S3 as DB.

armor009 commented 3 years ago

@badlop hoping to hear back from you.

Neustradamus commented 1 year ago

@armor009: There is now a PR:

cc: @RomanHargrave, @ArtemisPseudo, @licaon-kter.

armor009 commented 1 year ago

Thank you

RomanHargrave commented 1 year ago

Hey! Wow, I've been sucked into work a lot lately.

I am amenable to adding this to contrib, but I still feel strongly about getting into ejabberd directly. I'll be attending to holiday festivities this coming week, but would be interested in seeing what needs to be done to get this into contrib if it is desired.

badlop commented 1 year ago

I was curious, and did it following https://docs.ejabberd.im/developer/extending-ejabberd/modules/#add-your-module

It just required placing your files in new subdirectories and adding mod_s3_upload.spec. For completeness, I also added README, COPYING and an empty disabled config file. It's here: https://github.com/badlop/ejabberd-contrib/commits/s3_upload_draft

How to try it ?

  1. Apply this simple patch to ejabberd source code, to use this branch from my fork:
```diff diff --git a/src/ext_mod.erl b/src/ext_mod.erl index 1d8cb685e..b0adc7908 100644 --- a/src/ext_mod.erl +++ b/src/ext_mod.erl @@ -48,7 +48,7 @@ -include("translate.hrl"). -include_lib("xmpp/include/xmpp.hrl"). --define(REPOS, "https://github.com/processone/ejabberd-contrib"). +-define(REPOS, "https://github.com/badlop/ejabberd-contrib"). -record(state, {}). @@ -387,10 +387,10 @@ extract_github_master(Repos, DestDir) -> [$t,$i,$g,$.|T2] -> lists:reverse(T2); _ -> Base end, - case extract(zip, geturl(Url++"/archive/master.zip"), DestDir) of + case extract(zip, geturl(Url++"/archive/s3_upload_draft.zip"), DestDir) of ok -> RepDir = filename:join(DestDir, module_name(Repos)), - file:rename(RepDir++"-master", RepDir), + file:rename(RepDir++"-s3_upload_draft", RepDir), maybe_write_commit_json(Url, RepDir); Error -> Error @@ -746,7 +746,7 @@ maybe_write_commit_json(Url, RepDir) -> write_commit_json(Url, RepDir) -> Url2 = string_replace(Url, "https://github.com", "https://api.github.com/repos"), - BranchUrl = lists:flatten(Url2 ++ "/branches/master"), + BranchUrl = lists:flatten(Url2 ++ "/branches/s3_upload_draft"), {ok, _Headers, Body} = geturl(BranchUrl), {ok, F} = file:open(filename:join(RepDir, "COMMIT.json"), [raw, write]), file:write(F, Body), @@ -877,7 +877,7 @@ get_page(Node, Query, Lang) -> get_module_home(Module, Attrs) -> case element(2, lists:keyfind(home, 1, Attrs)) of - "https://github.com/processone/ejabberd-contrib/tree/master/" = P1 -> + "https://github.com/processone/ejabberd-contrib/tree/s3_upload_draft/" = P1 -> P1 ++ atom_to_list(Module); Other -> Other ```

  1. Compile and install ejabberd

  2. Get the updated ejabberd-contrib source code

    ejabberdctl modules_update_specs

  3. Compile and install the module

    ejabberdctl module_install mod_s3_upload

  4. Now it's time to configure it in ejabberd.yml or in the provided mod_s3_upload.yml

  5. Finally, restart ejabberd or simply reload the configuration

    ejabberdctl reload_config

Right now the tests are not used in the ejabberd-contrib action; I'll check that later.

I was able to compile and install the module. But I didn't test it works correctly.

Once you are able to test it, if it works correctly, you can use the content from this draft to prepare your PR.

RomanHargrave commented 1 year ago

@Neustradamus this is now in contrib

Neustradamus commented 1 year ago

@RomanHargrave: Good job, thanks, your PR has been merged!

Linked to:

Neustradamus commented 1 year ago

@RomanHargrave: Your PR in ejabberd has not been closed, it is time, it is now in ejabberd-contrib since 2 months:

@badlop: Several tickets too?

Cronoburn commented 2 months ago

I couldn't get this to work.. has something changed, or more likely I am doing it wrong? I'm not getting any errors, nor are the uploads showing up in my bucket. I've since tried using a python script to grab the upload from mod_http_upload, but that's throwing all sorts of errors.

licaon-kter commented 1 month ago

@Cronoburn better open a new issue with exact OS/ejabberd/config/etc.