Deploy to Heroku button stopped working

greglearns commented 5 years ago

The "Deploy to Heroku" button stopped working. It gives this output

-----> Apt app detected
-----> Detected Aptfile changes, flushing cache
-----> Adding custom repositories
-----> Updating apt caches
       Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
       Get:2 http://apt.postgresql.org/pub/repos/apt bionic-pgdg InRelease [46.3 kB]
       Get:3 http://apt.postgresql.org/pub/repos/apt bionic-pgdg/main amd64 Packages [263 kB]
       Get:4 http://archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
       Get:5 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
       Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
       Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1,344 kB]
       Get:8 http://archive.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [769 kB]
       Get:9 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 Packages [663 kB]
       Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,288 kB]
       Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [960 kB]
       Fetched 17.1 MB in 7s (2,463 kB/s)
       Reading package lists...
-----> Fetching .debs for gettext
       Reading package lists...
       Building dependency tree...
       0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 4 not upgraded.
       Need to get 1,293 kB of archives.
       After this operation, 0 B of additional disk space will be used.
       Get:1 http://archive.ubuntu.com/ubuntu bionic-security/main amd64 gettext amd64 0.19.8.1-6ubuntu0.3 [1,293 kB]
       Fetched 1,293 kB in 1s (1,489 kB/s)
       Download complete and in download only mode
-----> Installing gettext_0.19.8.1-6ubuntu0.3_amd64.deb
-----> Writing profile script
-----> Rewrite package-config files
-----> Rust app detected
-----> Downloading rustup
-----> Using rustup to install Rust channel stable
info: downloading installer
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2019-09-26, rust version 1.38.0 (625451e37 2019-09-23)
info: downloading component 'rustc'
info: downloading component 'rust-std'
info: downloading component 'cargo'
info: downloading component 'rust-docs'
info: installing component 'rustc'
info: installing component 'rust-std'
info: installing component 'cargo'
info: installing component 'rust-docs'
info: default toolchain set to 'stable'
  stable installed - rustc 1.38.0 (625451e37 2019-09-23)
Rust is installed now. Great!
To get started you need Cargo's bin directory (/app/tmp/cache/cargo/bin) in 
your PATH environment variable. Next time you log in this will be done 
automatically.
To configure your current shell run source /app/tmp/cache/cargo/env
-----> Building application using Cargo
    Updating crates.io index
 Downloading crates ...
  Downloaded base64 v0.10.1
  Downloaded env_logger v0.6.2
  Downloaded futures v0.1.28
  Downloaded lettre_email v0.9.2
  Downloaded idna v0.1.5
  Downloaded rand v0.7.0
  Downloaded toml v0.5.1
  Downloaded docopt v1.1.0
  Downloaded tokio-core v0.1.17
  Downloaded url v1.7.2
  Downloaded serde_json v1.0.40
  Downloaded serde v1.0.94
  Downloaded lettre v0.9.2
  Downloaded redis v0.10.0
  Downloaded native-tls v0.2.3
  Downloaded glob v0.2.11
  Downloaded openssl v0.10.23
  Downloaded hyper v0.11.27
  Downloaded hyper-staticfile v0.2.0
  Downloaded serde_derive v1.0.94
  Downloaded mustache v0.9.0
  Downloaded log v0.4.7
  Downloaded time v0.1.42
  Downloaded matches v0.1.8
  Downloaded lazy_static v1.3.0
  Downloaded hyper-tls v0.1.4
  Downloaded byteorder v1.3.2
  Downloaded base64 v0.9.3
  Downloaded quote v0.6.13
  Downloaded percent-encoding v1.0.1
  Downloaded iovec v0.1.2
  Downloaded tokio-service v0.1.0
  Downloaded scoped-tls v0.1.2
  Downloaded tokio-timer v0.2.11
  Downloaded openssl-sys v0.9.47
  Downloaded fast_chemail v0.9.6
  Downloaded log v0.3.9
  Downloaded itoa v0.4.4
  Downloaded regex v1.1.9
  Downloaded gettext v0.4.0
  Downloaded combine v3.8.1
  Downloaded httparse v1.3.4
  Downloaded atty v0.2.12
  Downloaded mime v0.3.13
  Downloaded relay v0.1.1
  Downloaded hostname v0.1.5
  Downloaded tokio-reactor v0.1.9
  Downloaded foreign-types v0.3.2
  Downloaded uuid v0.7.4
  Downloaded sha1 v0.6.0
  Downloaded proc-macro2 v0.4.30
  Downloaded cfg-if v0.1.9
  Downloaded net2 v0.2.33
  Downloaded want v0.0.4
  Downloaded openssl-probe v0.1.2
  Downloaded unicode-bidi v0.3.4
  Downloaded email v0.0.20
  Downloaded tokio-codec v0.1.1
  Downloaded language-tags v0.2.2
  Downloaded mio v0.6.19
  Downloaded humantime v1.2.0
  Downloaded bufstream v0.1.4
  Downloaded tokio-tcp v0.1.3
  Downloaded ryu v1.0.0
  Downloaded bytes v0.4.12
  Downloaded nom v4.2.3
  Downloaded unicase v2.4.0
  Downloaded rand_chacha v0.2.0
  Downloaded strsim v0.9.2
  Downloaded tokio v0.1.22
  Downloaded libc v0.2.60
  Downloaded termcolor v1.0.5
  Downloaded tokio-io v0.1.12
  Downloaded bitflags v1.1.0
  Downloaded tokio-executor v0.1.8
  Downloaded getrandom v0.1.6
  Downloaded syn v0.15.39
  Downloaded rand_core v0.5.0
  Downloaded futures-cpupool v0.1.8
  Downloaded tokio-proto v0.1.1
  Downloaded unicode-xid v0.1.0
  Downloaded memchr v2.2.1
  Downloaded version_check v0.1.5
  Downloaded num_cpus v1.10.1
  Downloaded tokio-uds v0.2.5
  Downloaded native-tls v0.1.5
  Downloaded ascii_utils v0.9.3
  Downloaded rand v0.4.6
  Downloaded c2-chacha v0.2.2
  Downloaded quick-error v1.2.2
  Downloaded aho-corasick v0.7.4
  Downloaded tokio-current-thread v0.1.6
  Downloaded tokio-udp v0.1.3
  Downloaded pkg-config v0.3.14
  Downloaded either v1.5.2
  Downloaded unicode-normalization v0.1.8
  Downloaded tokio-tls v0.1.4
  Downloaded utf8-ranges v1.0.3
  Downloaded tokio-fs v0.1.6
  Downloaded foreign-types-shared v0.1.1
  Downloaded autocfg v0.1.4
  Downloaded thread_local v0.3.6
  Downloaded regex-syntax v0.6.8
  Downloaded rand v0.6.5
  Downloaded parking_lot v0.7.1
  Downloaded encoding v0.2.33
  Downloaded tokio-threadpool v0.1.15
  Downloaded chrono v0.4.7
  Downloaded try-lock v0.1.0
  Downloaded spin v0.5.0
  Downloaded cc v1.0.37
  Downloaded safemem v0.3.0
  Downloaded tokio-sync v0.1.6
  Downloaded crossbeam-utils v0.6.5
  Downloaded unreachable v1.0.0
  Downloaded encoding-index-korean v1.20141219.5
  Downloaded ascii v0.9.2
  Downloaded crossbeam-deque v0.7.1
  Downloaded rand_hc v0.1.0
  Downloaded rand_pcg v0.1.2
  Downloaded fnv v1.0.6
  Downloaded smallvec v0.2.1
  Downloaded smallvec v0.6.10
  Downloaded slab v0.4.2
  Downloaded encoding-index-japanese v1.20141219.5
  Downloaded rand_core v0.4.0
  Downloaded rand_xorshift v0.1.1
  Downloaded take v0.1.0
  Downloaded rand v0.3.23
  Downloaded ucd-util v0.1.3
  Downloaded crossbeam-queue v0.1.2
  Downloaded rand_os v0.1.3
  Downloaded num-traits v0.2.8
  Downloaded slab v0.3.0
  Downloaded encoding-index-tradchinese v1.20141219.5
  Downloaded lock_api v0.1.5
  Downloaded parking_lot_core v0.4.0
  Downloaded rand_isaac v0.1.1
  Downloaded ppv-lite86 v0.2.5
  Downloaded encoding-index-singlebyte v1.20141219.5
  Downloaded mio-uds v0.6.7
  Downloaded lazy_static v0.2.11
  Downloaded num-integer v0.1.41
  Downloaded encoding-index-simpchinese v1.20141219.5
  Downloaded rand_jitter v0.1.4
  Downloaded rand_chacha v0.1.1
  Downloaded owning_ref v0.4.0
  Downloaded rustc_version v0.2.3
  Downloaded encoding_index_tests v0.1.4
  Downloaded void v1.0.2
  Downloaded crossbeam-epoch v0.7.1
  Downloaded rand_core v0.3.1
  Downloaded scopeguard v0.3.3
  Downloaded semver v0.9.0
  Downloaded stable_deref_trait v1.1.1
  Downloaded arrayvec v0.4.11
  Downloaded memoffset v0.2.1
  Downloaded semver-parser v0.7.0
  Downloaded nodrop v0.1.13
  Downloaded openssl v0.9.24
  Downloaded bitflags v0.9.1
   Compiling autocfg v0.1.4
   Compiling libc v0.2.60
   Compiling semver-parser v0.7.0
   Compiling rand_core v0.4.0
   Compiling byteorder v1.3.2
   Compiling spin v0.5.0
   Compiling arrayvec v0.4.11
   Compiling nodrop v0.1.13
   Compiling stable_deref_trait v1.1.1
   Compiling cfg-if v0.1.9
   Compiling log v0.4.7
   Compiling smallvec v0.6.10
   Compiling scopeguard v0.3.3
   Compiling proc-macro2 v0.4.30
   Compiling memoffset v0.2.1
   Compiling unicode-xid v0.1.0
   Compiling fnv v1.0.6
   Compiling futures v0.1.28
   Compiling slab v0.4.2
   Compiling syn v0.15.39
   Compiling cc v1.0.37
   Compiling pkg-config v0.3.14
   Compiling version_check v0.1.5
   Compiling bitflags v1.1.0
   Compiling ryu v1.0.0
   Compiling serde v1.0.94
   Compiling openssl v0.9.24
   Compiling openssl v0.10.23
   Compiling memchr v2.2.1
   Compiling foreign-types-shared v0.1.1
   Compiling native-tls v0.2.3
   Compiling scoped-tls v0.1.2
   Compiling bitflags v0.9.1
   Compiling httparse v1.3.4
   Compiling matches v0.1.8
   Compiling encoding_index_tests v0.1.4
   Compiling try-lock v0.1.0
   Compiling safemem v0.3.0
   Compiling itoa v0.4.4
   Compiling slab v0.3.0
   Compiling smallvec v0.2.1
   Compiling regex v1.1.9
   Compiling ppv-lite86 v0.2.5
   Compiling ascii_utils v0.9.3
   Compiling take v0.1.0
   Compiling openssl-probe v0.1.2
   Compiling void v1.0.2
   Compiling ucd-util v0.1.3
   Compiling lazy_static v0.2.11
   Compiling glob v0.2.11
   Compiling percent-encoding v1.0.1
   Compiling quick-error v1.2.2
   Compiling ascii v0.9.2
   Compiling redis v0.10.0
   Compiling language-tags v0.2.2
   Compiling utf8-ranges v1.0.3
   Compiling bufstream v0.1.4
   Compiling either v1.5.2
   Compiling strsim v0.9.2
   Compiling termcolor v1.0.5
   Compiling sha1 v0.6.0
   Compiling semver v0.9.0
   Compiling rand_chacha v0.1.1
   Compiling rand_pcg v0.1.2
   Compiling rand v0.6.5
   Compiling num-traits v0.2.8
   Compiling num-integer v0.1.41
   Compiling rand_chacha v0.2.0
   Compiling rand_core v0.3.1
   Compiling rand_jitter v0.1.4
   Compiling lazy_static v1.3.0
   Compiling owning_ref v0.4.0
   Compiling unicode-normalization v0.1.8
   Compiling tokio-sync v0.1.6
   Compiling tokio-service v0.1.0
   Compiling relay v0.1.1
   Compiling openssl-sys v0.9.47
   Compiling unicase v2.4.0
   Compiling nom v4.2.3
   Compiling email v0.0.20
   Compiling foreign-types v0.3.2
   Compiling unicode-bidi v0.3.4
   Compiling encoding-index-korean v1.20141219.5
   Compiling encoding-index-tradchinese v1.20141219.5
   Compiling encoding-index-simpchinese v1.20141219.5
   Compiling encoding-index-singlebyte v1.20141219.5
   Compiling encoding-index-japanese v1.20141219.5
   Compiling fast_chemail v0.9.6
   Compiling unreachable v1.0.0
   Compiling regex-syntax v0.6.8
   Compiling humantime v1.2.0
   Compiling portier_broker v0.2.0 (/tmp/build_fd0993635eb354f2190ffcf070d718b7)
   Compiling rustc_version v0.2.3
   Compiling rand_xorshift v0.1.1
   Compiling rand_hc v0.1.0
   Compiling rand_isaac v0.1.1
   Compiling crossbeam-utils v0.6.5
   Compiling thread_local v0.3.6
   Compiling c2-chacha v0.2.2
   Compiling lock_api v0.1.5
   Compiling idna v0.1.5
   Compiling encoding v0.2.33
   Compiling rand_os v0.1.3
   Compiling iovec v0.1.2
   Compiling net2 v0.2.33
   Compiling num_cpus v1.10.1
   Compiling rand v0.4.6
   Compiling getrandom v0.1.6
   Compiling time v0.1.42
   Compiling hostname v0.1.5
   Compiling atty v0.2.12
   Compiling parking_lot_core v0.4.0
   Compiling crossbeam-queue v0.1.2
   Compiling tokio-executor v0.1.8
   Compiling base64 v0.10.1
   Compiling base64 v0.9.3
   Compiling crossbeam-epoch v0.7.1
   Compiling log v0.3.9
   Compiling want v0.0.4
   Compiling quote v0.6.13
   Compiling aho-corasick v0.7.4
   Compiling combine v3.8.1
error: failed to run custom build command for `openssl v0.9.24`
Caused by:
  process didn't exit successfully: `/app/tmp/cache/target/release/build/openssl-c2f879b18e9e24ad/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Unable to detect OpenSSL version', /app/tmp/cache/cargo/registry/src/github.com-1ecc6299db9ec823/openssl-0.9.24/build.rs:16:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
warning: build failed, waiting for other jobs to finish...
error: build failed
 !     Push rejected, failed to compile Rust app.
 !     Push failed

stephank commented 5 years ago

Thanks for reporting this!

I'm not sure if there's a quick workaround, because I'm not super familiar with Heroku. It's this app.json file that determines the environment, and maybe there's something we can roll back there, like a buildpack? https://github.com/portier/portier-broker/blob/master/app.json

Otherwise, I suspect this is the old openssl bindings 0.9 causing an issue, which we indirectly depend on via an old Hyper 0.11. We may magically fix this by upgrading to Hyper 0.12, but there's also Hyper 0.13 and async/await around the corner, so I'm a bit hesitant to work on that angle until then.

greglearns commented 5 years ago

@stephank it does appear to be an openssl v0.9.24 problem. I just tried to update Portier-broker to use Hyper 0.12, but I gave up. In the mean time, that means that portier-broker cannot be deployed to Heroku.

Does anyone have thoughts on alternatives, since portier-broker cannot be deployed to Heroku? One option is that I could temporarily point at https://portier-demo.herokuapp.com/ for the time being since the site doesn't get much traffic currently, BUT, in order to verify / decode the JSON Web Token from the public instance, I need the public_key... is it possible to get that?

e.g. when I run my own broker, I generated my own public/private key, and then use the public key to decode/verify the token (see code below). However, if if I use https://portier-demo.herokuapp.com/ then I need the public key:

    decode::<Claims>(&token, include_bytes!("../../public_key.der"), &validation)

used in this context...

pub fn verify_token(
    token: String,
    _iss: String,
    _aud: String,
    _leeway: i64,
) -> Result<UserEmail, Error> {
    let validation = Validation {
        algorithms: vec![jsonwebtoken::Algorithm::RS256],
        iss: Some("https://portier-demo.herokuapp.com/".to_string()),
        leeway: 60,
        validate_exp: true,
        ..Default::default()
    };

    decode::<Claims>(&token, include_bytes!("../../public_key.der"), &validation)
        .map(|data| {
            dbg!(&data);
            Ok(UserEmail::from(data.claims))
        })
        .map_err(|_err| {
            dbg!(&_err);
            format_err!("ServiceError::Unauthorized")
        })?
}

stephank commented 5 years ago

We also deploy the public broker to Heroku, but using git. I tried to redeploy, and ran into the same error. Downgrading the stack to heroku-16 helps, though, and it looks like that stack is supported until somewhere in 2021, which should be fine.

It looks like we can pin the stack in app.json simply by setting stack there. Could you try using this deploy link? https://heroku.com/deploy?template=https://github.com/portier/portier-broker/tree/fix/heroku

As for the keys, we follow OpenID connect, so you can actually fetch those keys as JWK by fetching /.well-known/openid-configuration, then following the URL in the jwks_uri property of that document. (Both of these can be cached according to HTTP headers.)

Keys are intended to rotate. That currently doesn't happen for most of our deploys, unfortunately. But when we get around to fixing it, hardcoding keys is going to break.

greglearns commented 5 years ago

1) The Heroku button works now. Great! Thank you! 2) Having just deployed the broker to Heroku, it will use the private key I created and pasted into BROKER_KEYTEXT. But when you deployed broker.portier.io, do you have to do the same thing? (I'm assuming the answer is "yes", but I just want to confirm.) 3) For verifying JWKs from the new instance I launched to Heroku or from broker.portier.io, I don't see how to get public key from the /keys.json (which the jwks_uri property in /.well-known/openid-configuration points to), because keys.json has the public components (n and e), but not the actual public key. Using Rust, I can't figure out how to validate the broker.portier.io JWKs because https://github.com/Keats/jsonwebtoken does not support generating the public key from the public components (which broker.portier.io/keys.json exposes) except in the experimental https://github.com/Keats/jsonwebtoken/tree/next branch that has some other issues. Is there a recommended way in Rust to validate the JWKs generated by broker.portier.io, or from the new Heroku instance I just deployed?

And, if there is an answer, I couldn't find it in the documentation. I can help put it into the documentation if needed.

stephank commented 5 years ago

Yes, it's the same.
Oh, that's right. I looked into jsonwebtoken before, and ran into the same problem. I'm not sure the library support is there in Rust. :/ For now, your solution of including the public key will work fine, I guess.

greglearns commented 5 years ago

That's a bummer about not being able to support Rust! But, I just found this -- is this the answer or is this for something else? https://github.com/portier/portier-broker/blob/master/src/crypto.rs#L188

stephank commented 5 years ago

Oh, that's right! I guess I was confused. But that's exactly the code we use to verify a token signature.

And this, specifically, is how we parse a JWK: https://github.com/portier/portier-broker/blob/a0b3daed7c8c65b1c92b06328bd7d65accf1a3eb/src/crypto.rs#L180-L183

I guess we don't use any libraries to abstract it, though, just plain openssl.

(I think I was maybe confused about trying to use the ring crate before, and having trouble using that. The jsonwebtoken crate also uses ring instead of openssl, IIRC.)

greglearns commented 5 years ago

That's right that the jsonwebtoken crate uses ring. So, basically, I can just lift that code from crypto.rs and use that it sounds like. Thank you. (but, I'm curious, why do this crate need to verify a token, since I thought it just generates them?)

stephank commented 5 years ago

I think that should work, yes.

The broker has to verify tokens from identity providers. The protocol between client -> broker and broker -> idp is roughly the same.

portier / portier.github.io

Deploy to Heroku button stopped working #40