Closed batbrain7 closed 5 years ago
In the recent progress with the Fdroid issue I successfully generated the metadata file, passed all the lint tests and started the local build of fdroidserver with susi.ai added in it. Three errors were faced by me :
The first error is with the link preview library for which alternative can be easily found, second error is with the binary file that is put in the app so that the HotWordDetection can take place.
So as we can see above that build was successful but these errors came in way. So once these errors are fixed we can make a merge request to F-Droid
To check for Fdroid compatibility and to build the app using fdroid follow these blogs :
For the second and third file we need to find an alternative to provide hotword detection and replace these binary files or another approach would be download these files after the app is installed.
There are not many good options for the download. So, downloading after installation sounds good to me.
@batbrain7 I'm not sure if binary files can be downloaded after installation. Also, this may violate the terms of service of FDroid
Snowboy is open source, so look for how this can be integrated in the build process
@batbrain7 @iamareebjamal These two ".so" files are created using SWIG. I tried finding out the C or C++ file that could build the .so files using NDK, but that also won't help as these shared object files will be again causing the issues.
How is downloading going to work? Please link to a documentation or article which provides more info into that. I may be wrong but I don't think it is possible to download and dynamically link the binary files like that
@iamareebjamal I did research it on the internet, I thought just like the obb package files we could download these files too, but I too don't think it is possible. https://android.jlelse.eu/app-bundles-and-dynamic-feature-modules-a-silver-bullet-for-bloated-apks-d0bcac151ae6 This article shows dynamic bundles but then again it requires play core library so it can't be used.
Even that doesn't allow you to download what you want. It downloads stuff automatically, the resources, layouts and libs it needs
@iamareebjamal One suggestion I recieved on the post is
The README you linked contains instructions how to build the native libraries. Put them into the build recipe.
Please guide me for the same, the instructions given are here: https://github.com/Kitt-AI/snowboy/blob/master/examples/Android/README.md
@iamareebjamal I researched that it uses swig and compile all the native methods in two .so files, but the problem here is how can I include in the build process? so that it creates that files during the build and not before hand.
Can you do it in F-Droid build process and not the SUSI build?
I think they are referring to build recipe(F-Droid) and not the build process of SUSI because I don't know how that will change the matter.
In any way we can change the process is to maintain a fork of snowboy which is not recommended. Clarify please
@iamareebjamal @batbrain7
1 actionable task: 1 executed
INFO: Scanning source for common problems...
ERROR: Found usual suspect 'youtube.*android.*player.*api' at app/libs/YouTubeAndroidPlayerApi.jar
WARNING: Found JAR file at app/libs/YouTubeAndroidPlayerApi.jar
WARNING: Found possible binary at app/src/main/assets/snowboy/alexa_02092017.umdl
WARNING: Found possible binary at app/src/main/assets/snowboy/common.res
ERROR: Found shared library at app/src/main/jniLibs/arm64-v8a/libsnowboy-detect-android.so
ERROR: Found shared library at app/src/main/jniLibs/armeabi-v7a/libsnowboy-detect-android.so
INFO: Removing gradle-wrapper.jar at gradle/wrapper/gradle-wrapper.jar
ERROR: Could not build app ai.susi: Can't build due to 3 errors while scanning
INFO: Finished
INFO: 1 build failed
it seems even after adding playStoreimplementation
the youtube library is scanned
No, you should be building the fdroid variant and not a general build
The most recent problem that was faced is how to build a particular flavor in the fdroid build process, so I posted the issue on the forum : https://forum.f-droid.org/t/how-to-build-a-particular-app-flavor-on-fdroid/3346
INFO: Removing gradle-wrapper.jar at gradle/wrapper/gradle-wrapper.jar
INFO: Creating source tarball...
ERROR: Could not build app ai.susi due to unknown error: Traceback (most recent call last):
File "/usr/local/Cellar/fdroidserver/1.0.9/libexec/lib/python3.7/site-packages/fdroidserver/build.py", line 1120, in main
options.onserver, options.refresh):
File "/usr/local/Cellar/fdroidserver/1.0.9/libexec/lib/python3.7/site-packages/fdroidserver/build.py", line 896, in trybuild
build_local(app, build, vcs, build_dir, output_dir, log_dir, srclib_dir, extlib_dir, tmp_dir, force, onserver, refresh)
File "/usr/local/Cellar/fdroidserver/1.0.9/libexec/lib/python3.7/site-packages/fdroidserver/build.py", line 562, in build_local
tarball.add(build_dir, tarname, exclude=tarexc)
TypeError: add() got an unexpected keyword argument 'exclude'
INFO: Finished
INFO: 1 build failed
Now I am able to build the app on fdroid after removing the files but right after that I get this error
Always share your metadata reference here with the build log
You must be on Python 3.7, should be on 3.6
You seem to have installed fdroidserver via apt, if you had downloaded the git repo, you could have changed the code, easiest way would be to install fdroidserver in a virtualenv if possible
exclude was removed in Python 3.7 and replaced with filter
Categories:
- Internet
License: Apache-2.0
SourceCode: https://github.com/fossasia/susi_android
IssueTracker: https://github.com/fossasia/susi_android/issues
Summary: Susi AI is an intelligent personal assistant
Description: Susi AI is an intelligent Open Source personal assistant. It is fully
customizable and developed by a community of developers. It is capable of chat
and voice interaction by using APIS to perform actions such as music playback,
making to-do lists, setting alarms, streaming podcasts, playing audiobooks, and
providing weather, traffic, and other real-time information. Additional functionalities
can be added as console services using external APIs. Susi AI is able to answer
questions and depending on the context will ask for additional information in
order to perform the desired outcome. The core of the assistant is the Susi AI
server that holds the "intelligence" and "personality" of Susi AI. The Android
and web applications make use of the APIs to access information from a hosted
server.
RepoType: git
Repo: https://github.com/fossasia/susi_android
Builds:
- versionName: 1.0.10-Fdroid
versionCode: 11
commit: 1ad2fd0e858b1256617e652c6c8ce1b8372473e6
subdir: app
gradle:
- fdroid
rm:
- app/src/main/jniLibs/arm64-v8a/libsnowboy-detect-android.so
- app/src/main/jniLibs/armeabi-v7a/libsnowboy-detect-android.so
- app/libs/YouTubeAndroidPlayerApi.jar
AutoUpdateMode: None
UpdateCheckMode: Tags
this is the metadata reference file used for the build
@iamareebjamal can I move down a version in python?
dyld: Library not loaded: @executable_path/../.Python
Referenced from: /usr/local/Cellar/fdroidserver/1.0.9/libexec/bin/python3.7
Reason: image not found
Abort trap: 6
on moving down to version 3.6.5 I get this error , @iamareebjamal unable to understand this
Your fdroid is linked to python 3.7
You have to reinstall it
brew install fdroidserver
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> Updated Formulae
vips
==> Installing dependencies for fdroidserver: python
==> Installing fdroidserver dependency: python
==> Downloading https://homebrew.bintray.com/bottles/python-3.7.0.sierra.bottle.
######################################################################## 100.0%
==> Pouring python-3.7.0.sierra.bottle.1.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
rm '/usr/local/bin/2to3'
To force the link and overwrite all conflicting files:
brew link --overwrite python
To list all files that would be deleted:
brew link --overwrite --dry-run python
Possible conflicting files are:
/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.6/bin/2to3
/usr/local/bin/idle3 -> /Library/Frameworks/Python.framework/Versions/3.6/bin/idle3
/usr/local/bin/pydoc3 -> /Library/Frameworks/Python.framework/Versions/3.6/bin/pydoc3
/usr/local/bin/python3 -> /Library/Frameworks/Python.framework/Versions/3.6/bin/python3
/usr/local/bin/python3-config -> /Library/Frameworks/Python.framework/Versions/3.6/bin/python3-config
/usr/local/bin/pyvenv -> /Library/Frameworks/Python.framework/Versions/3.6/bin/pyvenv
==> /usr/local/Cellar/python/3.7.0/bin/python3 -s setup.py --no-user-cfg install
==> /usr/local/Cellar/python/3.7.0/bin/python3 -s setup.py --no-user-cfg install
==> /usr/local/Cellar/python/3.7.0/bin/python3 -s setup.py --no-user-cfg install
==> Caveats
Python has been installed as
/usr/local/bin/python3
Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
`python3`, `python3-config`, `pip3` etc., respectively, have been installed into
/usr/local/opt/python/libexec/bin
If you need Homebrew's Python 2.7 run
brew install python@2
Pip, setuptools, and wheel have been installed. To update them run
pip3 install --upgrade pip setuptools wheel
You can install Python packages with
pip3 install <package>
They will install into the site-package directory
/usr/local/lib/python3.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python
==> Summary
🍺 /usr/local/Cellar/python/3.7.0: 4,787 files, 102MB
@iamareebjamal brew always installs fdroid linked with python 3.7
OK, then simply clone the git version, remove brew one. It's not gonna work
Also, remove the application ID suffix, it's not gonna work
Could not build app ai.susi: Wrong package ID - build ai.susi.fdroid but expected ai.susi
how to build the app after uninstalling the brew one as the "fdroid build" command is not available now
Did you install using git?
I have forked the fdroid repository into my own and then cloned the forked repository
Yeah, so there is fdroid binary in it, you must not have added it in your path
@batbrain7 It's been 2 days. I am able to build SUSI on my PC since we last spoke. What's the issue?
@iamareebjamal since the PR you made now if I install the fdroidserver with brew will it work ?
PR is not merged yet and even if it was, brew doesn't update libraries that fast
Why are you not cloning the repo, it's literally much much easier. Ask @nikit19 if you have doubts
@iamareebjamal I have cloned the respo but I don't understand where to add it in the path
@batbrain7
I did the following to install the server
git clone https://gitlab.com/fdroid/fdroidserver.git
export PATH="$PATH:$PWD/fdroidserver"
credits: https://gitlab.com/fdroid/fdroiddata/blob/master/README.md#quickstart
this gives me error @nikit19 @iamareebjamal :
Traceback (most recent call last):
File "/Users/mohitkumar/Documents/fdroidserver/fdroid", line 24, in <module>
import fdroidserver.common
File "/Users/mohitkumar/Documents/fdroidserver/fdroidserver/common.py", line 50, in <module>
from pyasn1.codec.der import decoder, encoder
ModuleNotFoundError: No module named 'pyasn1'
You have to install it
https://forum.f-droid.org/t/python-no-module-named-pyasn1-modules/2991/5
Found distributionUrl=https://services.gradle.org/distributions/gradle-4.7-all.zip via distributionUrl
/Users/mohitkumar/Documents/fdroidserver/gradlew-fdroid: line 59: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
/Users/mohitkumar/Documents/fdroidserver/gradlew-fdroid: line 109: 1.4: syntax error: invalid arithmetic operator (error token is ".4")
No hash for gradle version distributionUrl=https://services.gradle.org/distributions/gradle-4.7-all.zip! Exiting...
ERROR: Could not build app ai.susi: Error cleaning ai.susi:1.0.10
==== detail begin ====
Found distributionUrl=https://services.gradle.org/distributions/gradle-4.7-all.zip via distributionUrl
/Users/mohitkumar/Documents/fdroidserver/gradlew-fdroid: line 59: declare: -A: invalid option
declare: usage: declare [-afFirtx] [-p] [name[=value] ...]
/Users/mohitkumar/Documents/fdroidserver/gradlew-fdroid: line 109: 1.4: syntax error: invalid arithmetic operator (error token is ".4")
No hash for gradle version distributionUrl=https://services.gradle.org/distributions/gradle-4.7-all.zip! Exiting...
==== detail end ====
INFO: Finished
INFO: 1 build failed
After building it, I get this error? @iamareebjamal @nikit19 any ideas?
This is because you are using mac AFAIK
Are you using bash on it? If not, install it
This is why it is requested to contributors to stay at same environment, the declare
command is different in bash and macOS
The script won't run properly
after installing bash too, I get the same error , are the steps different after installing bash than usual?
bash is a shell, did you run the instructions through bash after installing it? This is basic requirements setup
@iamareebjamal even after running all the steps through the bash shell, still I get the same error
What did you do?
@batbrain7 This is taking too long, what should we do?
@iamareebjamal I installed bash on my machine :
here is how it looks after opening it, now on reinstalling the fdroid server i.e cloning the fdroidserver again and following all the steps with this I get the above error again, is this not the bash to be used ?
Please setup TeamViewer and share your credentials in private Gitter channel ASAP so I can take a look into the issue, have the fdroiddata and server directory open with a terminal as well
We should tackle this within this day
Okay @iamareebjamal shared the credentials please check
@batbrain7 I did the following to install the server git clone https://gitlab.com/fdroid/fdroidserver.git export PATH="$PATH:$PWD/fdroidserver"
credits: https://gitlab.com/fdroid/fdroiddata/blob/master/README.md#quickstart
@nikit19 @iamareebjamal I have cloned repo from the original fdroid repo and not mine the forked, how to send MR now ?
Currently susi is only on play store, we have to launch it on the F-Droid .