Open buncis opened 3 years ago
apparently this is because default python that heroku use is not builded with --enable-shared
thus why our build is failed because pycall.rb
need python
that use enable-shared
to fix this problem here are the steps
since using pycall.rb our apps is consist of multiple stack (Python, Ruby, Javascript) to set the buildpack what you need to do are
clean the current buildpack
heroku buildpacks:clear
add the needed buildpack with order (-i set the order)
heroku buildpacks:add heroku/python - i 1
heroku buildpacks:add heroku/nodejs -i 2
heroku buildpacks:add heroku/ruby -i 3
heroku sometimes use build cache instead compiling from scratch, to clear it
https://help.heroku.com/18PI5RSY/how-do-i-clear-the-build-cache
first install the plugin
heroku plugins:install heroku-builds
Then use the following command to clear the cache:
heroku builds:cache:purge
The cache will be rebuilt on the next deploy. If you do not have any new code to deploy, you can push an empty commit.
$ git commit --allow-empty -m "Purge cache"
$ git push heroku master
the python buildpacks from heroku need two file to run, build and install the dependencies
runtime.txt
this is file where u declared ur python version
my runtime.txt
python-3.8.10
requirements.txt
this is file where u declare python package that you use
my requirements.txt
openpyxl==3.0.7
numpy==1.21.1
pandas==1.3.1
put those 2 files on root of your rails app folder, they are on same folder with your Gemfile
and .ruby-version
--enable-shared
configthis is essentials since our beloved pycall.rb
depends on it, it cannot work with python without --enable-shared
there is multiple way to achieve this, the readme page has it owns section, but imho its kinda obsolete and not working
is using the built in post_compile
hooks from official heroku/python buildpacks
to do that add post_compile
to your bin/
folder in your rails app
post_compile
set -e
buildpack_url=https://github.com/heroku/heroku-buildpack-python
buildpack_vsn=v197
# rebuild python if it's missing enable-shared
if ! python3 -msysconfig | grep enable-shared \
> /dev/null; then
PYTHON_VERSION="$(< runtime.txt)"
git clone -b "$buildpack_vsn" "$buildpack_url" _buildpack
export WORKSPACE_DIR="$PWD/_buildpack/builds"
rm -f .heroku/python/bin/python # prevent failing ln after build
sed -i 's!figure --pre!figure --enable-shared --pre!' \
"$WORKSPACE_DIR"/runtimes/python3
"$WORKSPACE_DIR/runtimes/$PYTHON_VERSION" /app/.heroku/python/
rm -fr _buildpack
fi
with this when the heroku building your app for first time it will check did it have your required python runtime if not then they will install it but they will install it using pre-compile python that not use the --enable-shared
options
after this first time script run then it will call the post_compile
script
the post_compile
script
will check if the installed python is have --enabled-shared
config
if not the script will tell to remove the current installed python and compile the python runtime with --enabled-shared
after this your app could build smoothly
is to compile the python by urself with --enable-shared
and upload the precompiled python somewhere then fork the official https://github.com/heroku/heroku-buildpack-python
buildpacks and somehow make the scripts aim to download your precompiled python instead using heroku precompiled python
if you want to check if your current python runtime has --enable-shared you could do run
heroku run bash
then paste this
$ python3 -msysconfig | grep enable-shared
if there is --enable-shared then you good to go
another approach is in your bash open python then from your python repl
import sysconfig
sysconfig.get_config_vars('Py_ENABLE_SHARED')
if it return [1]
then its builded with --enable-shared
@mrkn thanks for this gem with this gem I could get the best of two worlds python pandas and numpy + ruby and rails
could we add this into wiki or readme.md
?
but I think putting this on readme
would be too much
@buncis I'm sorry for my response to being late. PyCall's README already has the section that describes tips for deploying on Heroku. This section was added by the pull-request https://github.com/mrkn/pycall.rb/pull/134 on March 9, 2021. Did you check this part in README?
@mrkn I have read that readme before, problem is I can't make my app work with that README
I think we could close this issue right now and could you enable discussion or wiki in this github repo?
so I could post my issue and guide to there?
@buncis I don't want to open the wiki because the wiki is too difficult to keep managed. Opening the discussion can be acceptable, but I believe README is appropriate for mentioning the way to resolve this problem.
I want to know what information you needed is absent in the current README. Could you please tell me about it?
all the buildpacks mentioned in the readme is too old for newer applications or just doesn't works because its not updated
the only solution is to make ur own buildpacks or recompile the heroku default buildpack with --enable-shared options
On Thu, Feb 10, 2022, 10:15 AM Kenta Murata @.***> wrote:
@buncis https://github.com/buncis I don't want to open the wiki because the wiki is too difficult to keep managed. Opening the discussion can be acceptable, but I believe README is appropriate for mentioning the way to resolve this problem.
I want to know what information you needed is absent in the current README. Could you please tell me about it?
— Reply to this email directly, view it on GitHub https://github.com/mrkn/pycall.rb/issues/151#issuecomment-1034449214, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADMZEW34UC6JDFZWHKDLVKDU2MUWHANCNFSM5BI2A4IA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: <mrkn/pycall .@.***>
@buncis Thanks. I understand. It should be described in README and I want to do so. Could you please make a pull request to update the Heroku section in README?
I wish, but I can't do it right now, I'll do it later
@buncis No problem. Thank you very much!
when I want to deploy to heroku
the application always failed to build with this error message
heroku logs -t