[Deadline: Feb 1st, 2022] JCenter is being sunset -- What's the plan for react-native?

fbartho commented 3 years ago

Introduction

Disclaimer: I'm neither an Android expert nor do I know the right terminology here

JCenter / Bintray are going away, and this will have an impact on react-native Android (Core), and probably all react-native Android apps.

JCenter / Bintray is a service for build artifacts for Java projects, and jcenter() can be found in the build.gradle and other locations for the default react-native template.

Details

Deadline: Feb 1st, 2022
Announcement: https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/
Search of react-native's codebase for JCenter

Discussion points

What's the impact of this change? -- my guess: mass disruption to all android react-native apps
Are there alternatives to JCenter for the react-native community?
What's the plan for migrating react-native repos? -- depends on what maintainers / facebook plan to do?
What's the plan for communication with the community?

kelset commented 3 years ago

I think that the quick answer is that:

we'll have to do PRs against the main repo to remove all references to jcenter() from gradle configs (probably in favour of mavenCentral()...? ). And see what happens.
existing React Native apps will have to do a similar process on their own.
we'll have to update the template to not have jcenter() in the build.gradle too.

EDIT: me and my colleagues tried to dive a bit more into this problem and it sounds like either it's gonna an easy fix or a mess.

fbartho commented 3 years ago

Thanks for taking a look @kelset -- I'm expecting it to be a whole mess too!

As I'm not deep in the android ecosystem, I don't even know if the jcenter() call is actually reaching out to a "host" of the content, and if mavenCentral() is: A. a different host B. a superset of JCenter(), or C. something else.

I have some dozens of android/ folders in my node_modules libraries, so even if I update my app to use mavenCentral() I don't know what will happen to all the libraries. -- It's pretty much guaranteed that all the libraries won't ship an update in the next month or two, so… This is going to be fun.

Forgive my lack of name recognition, but are you "the powers that be" in the react-native community, or do you know how to escalate this?

kelset commented 3 years ago

yeah it's likely going to be a wild ride 🤠

Forgive my lack of name recognition, but are you "the powers that be" in the react-native community, or do you know how to escalate this?

Ahahah no worries at all - I think it's a fair question! I don't work for FB but I think that posting it here is a great start, I am in the "RN Contributors" discord so I'll try to put this in front of the team and see what they say (in all fairness, it has already been reported there by @empyrical ;) )

Naturalclar commented 3 years ago

Update about the deadline, it seems like JFrog has extended the jcenter sunset. They'll be serving packages until February 1st, 2022

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

pke commented 3 years ago

Except jcenter are down right now. This again proves the point that build pipelines should have zero external dependencies. Anyone knows how to setup a company wide maven-repo that it manually curated?

mifi commented 3 years ago

Yea, they say All systems operational at https://status.bintray.com/ but packages have been sporadically returning 502 bad gateway for over a day now. I think it's time to switch!

Naturalclar commented 3 years ago

I believe we're waiting for a new version of gradle which doesn't depend on jcenter, gradle 7.0. https://blog.gradle.org/jcenter-shutdown

Can't do much on react-native's (or any other cross platform's) side until that is resolved

kelset commented 3 years ago

Small update on this, we have found a potential workaround on our side, you can see it in this PR: https://github.com/microsoft/react-native-test-app/pull/278

Basically we point the old trove4j dependency to the new IntelliJ one with a local script.

mikehardy commented 3 years ago

@Naturalclar "I believe we're waiting for a new version of gradle..."

I do not understand ? The dependencies of react-native (direct or transitive) must be published in non-jcenter repositories, and you remove jcenter() from your build scripts, and you are "jcenter clean", how is a new version of gradle going to publish the artifacts to other repositories for you 🤔 ?

There's real work to be done, quite specifically re: yog'a proguard-annotations artifact. I don't think waiting on anything is a good plan.

That said, behold: gradle 7 is here! So I guess if that was some reason for waiting, there's no reason to wait.

Copy-splatted in here from https://github.com/react-native-community/releases/issues/221#issuecomment-822024961

This is just context but I did a deep dive on "jcenter/bintray is going away" to see if I could purge react-native-device-info module of jcenter uses - including the example app and it appears yoga's 'proguard-annotations' artifact is the key to it all. There are 3 items total

1- flipper itself (requested above) 2- flipper dependency on fbjni-android-only (fixed in flipper 0.79+, so we actually need a newer one than is even on main dev branch but wait...) 3- flipper dependency on yoga progaurd-annotations which is still published on jcenter only but tracked here and will require a still-newer flipper unless they publish old artifacts on jitpack or mavenCentral

mikehardy commented 3 years ago

(there's separate discussion here about whether modules will update - I can't see a way around it, if they don't people will have to fork and publish to jitpack until the repo is updated - and if it's not updated for such an important issue then repo should be forked, IMHO)

mikehardy commented 3 years ago

Had a brainstorm on a solution that could be made to work similar to jetifier - a small centralized solution that adds forward-compatibility to projects dependent on old versions and can't upgrade for whatever reason. Realized after posting it that it would be better here:

https://github.com/facebook/yoga/issues/1065#issuecomment-823283295

@d4vidi that's an interesting approach, you are on to something there. This is a potential start towards something automated for people that can't update.

Examine this if you have time: https://github.com/ankidroid/Anki-Android/blob/master/AnkiDroid/robolectricDownloader.gradle - the idea is to pre-download maven artifacts you'll need later

I wrote that in order to have external dependency download de-flaked in an E2E CI type pipeline where the downloads were failing builds and giving me a fail signal outside of my code changes. With that and an exponential-back-off-retry I have great signal to noise.

But - if the general idea of "you can have a gradle task that downloads dependencies before your real build" is combined with the idea of "react-native jcenter dependencies may be downloaded"...

1- store the problematic in-use artifacts on the official react-native github somewhere (or on their respective project pages) as downloadable things - have a known list of them, I think there are only a handful (yoga android-annotations, 2 flipper plugins, fbjni)

2- write a downloader script for them like the above

3- integrate that in to patch releases of RN CLI?

It could be just as terrible + functional as https://github.com/mikehardy/jetifier :-), which is and was kind of gross but also just glossed over the whole AndroidX migration for everyone at once and made it go away...

If done well it requires no external approval or waiting actually - a repo could simply get them now even if it wouldn't be officially blessed, write the task and throw them in the local ~/.m2 and integration of the package is to add an apply ... statement in your build script (something semi-normal for packages to ask) and make sure mavenLocal is in your repo list in build.gradle, so as a proof-of-concept it could be demonstrated prior to involving the official repos (so the artifacts are more official) or the RN CLI (so the task is included / default-on like jetifier for everyone)

Note that it may be useful to develop this capability regardless of the dependencies we know about being fixed, as there is always some other thing we did not know about. And if we have this capability then someone somewhere can get a copy of the artifact, post it in the downloadable spot, we can add it to the npm package for an update to the list and everyone is auto-fixed again until versions are actually updated (similar to jetifier which did have some missing edge cases at the start until adoption was 100% and is still a lifesaver for folks on RN59 etc)

mikehardy commented 3 years ago

I'm going to try it. Everything's more fun with more people, come collaborate if you want to try it with me: https://github.com/mikehardy/jcenterizer/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-asc

d4vidi commented 3 years ago

@mikehardy I might be able to help at some point on Thursday. 'Promise to approach you.

d4vidi commented 3 years ago

@mikehardy 'ended up having a hectic Thursday. Have you managed to make progress so far?

mikehardy commented 3 years ago

I totally understand @d4vidi my country just went back in to lockdown while we were on a business trip within the country and it scrambled my week completely. At any rate I don't have code to show yet - it's still at the research phase but

1) your idea for it to be a gradle plugin is perfect I think, the whole idea is a "shim" basically, and it means we would be isolating the shim right at the layer we are shimming using the same tools etc. Now that I've seen that idea I think any solution that is not a gradle plugin will need to have a strong reason why

2) having a "known-good download location" for orphaned artifacts is one way to handle things, but @kelset pointed me to another method also where you may force resolution of artifacts to ones that are generally available https://github.com/microsoft/react-native-test-app/pull/278/files#diff-ddc2bac746c108f9464843bb2c54869f66913b422daa4df1faa0a2ec1a1d7a2f

I don't see those two methods (maintain access to identical copies of orphaned artifacts, maybe substitute compatible artifacts) as in competition at all personally I think both methods are clever and will be useful and I'd like to have both be possible. For the "reproducible build" crowd they'll want SHAs to match up with the original artifacts on whatever code they can't change without re-qualifying I think, so both ideas have room in any solution

d4vidi commented 3 years ago

@mikehardy I'm familiar with the approach and have even utilized it in the context of the com.facebook.fbjni:fbjni-java-only artifact, due to dep discrepancies of versions 0.0.3 vs. 0.0.4 (it is indirectly required by RN itself).

The technique is perfectly valid for artifacts that are both hosted and which hosted version plays well with the depending libraries, in terms of compatibility. However, this isn't always the case, as we've seen with respect to com.facebook.yoga:proguard-annotation, which doesn't seem to be hosted anywhere. For that, you are forced to host the artifacts elsewhere, and this is where you suggestion comes in to play as everyone's life saver.

mikehardy commented 3 years ago

Yeah I see it as one trick in a bag of tricks. Finally got enough react native firebase stuff off the queue last week I should be able to do a poc today 🤞

mikehardy commented 3 years ago

While doing some research on this, this morning, I re-checked and saw this:

UPDATE 4/27/2021: We listened to the community and will keep JCenter as a read-only repository indefinitely. Our customers and the community can continue to rely on JCenter as a reliable mirror for Java packages.

https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

So as far as I am concerned this is resolved -> wontfix, though I have ideas posted up in my repo if needed.

kelset commented 3 years ago

We listened to the community and will keep JCenter as a read-only repository indefinitely.

Oh wow that's huge. Yeah I guess we can close this one off since it won't need an actual fix ever, then :D

dmoebius commented 3 years ago

JCenter will be put into read-only mode. Where will you publish updates?

kelset commented 3 years ago

@dmoebius I'm not sure what you are referring to; the only package that is "stuck" in the JCenter limbo is org.jetbrains.trove4j:trove4j and that's already an unmaintained package, so there have not been updates to it in a very long time.

dmoebius commented 3 years ago

Ok, but are there any plans to get rid of the unmaintained dependency trove4j in the long term?

And what about com.facebook.yoga:proguard-annotations? AFAICT react-native directly depends on it (see https://github.com/facebook/yoga/issues/1065#issuecomment-776926596).

And the Flipper support built-in to react-native since 0.62 depends on Fresco, which is currently available on JCenter only (at least the version RN uses, see https://github.com/facebook/fresco/issues/2585).

kelset commented 3 years ago

Iiinteresting, I wasn't aware. Let's keep this open until we get some feedback from FB folks on this then.

mikehardy commented 3 years ago

It is my understanding that since JCenter will exist, those old versions are fine to depend on for old versions of react-native. The new versions of everything except proguard-annotations are already elsewhere.

For proguard-annotations specifically, that seems like it should be filed in the yoga repo (I think it already is?) and then PR'd back to other repos. But it's more about "forward progress" now than "oh nooooo...." which was it's previous state

flutterq commented 3 years ago

[Solved] Please remove usages of jcenter() Maven repository from your build scripts / JCenter is at end of life

tijs commented 3 years ago

Isn't it a simple change for the maintainers of proguard-annotations though? The recommendation for android projects is to remove the jcenter() repo and instead use the mavenCentral() repo. Package maintainers have been asked to move their packages to mavenCentral quite a while back.

As a React Native developer, if i want to get rid of the jcenter warnings i only have to replace jcenter() with mavenCentral() in my build.gradle file.

Currently this works for all our dependencies, in a very large project, except for com.facebook.yoga:proguard-annotations which has not been moved to mavenCentral yet. So it would be kinda nice to have annotations move as well so i can clean up my project config and have one less repo in my build file.

pvegh commented 3 years ago

Isn't it a simple change for the maintainers of proguard-annotations though? The recommendation for android projects is to remove the jcenter() repo and instead use the mavenCentral() repo. Package maintainers have been asked to move their packages to mavenCentral quite a while back.

As a React Native developer, if i want to get rid of the jcenter warnings i only have to replace jcenter() with mavenCentral() in my build.gradle file.

Currently this works for all our dependencies, in a very large project, except for com.facebook.yoga:proguard-annotations which has not been moved to mavenCentral yet. So it would be kinda nice to have annotations move as well so i can clean up my project config and have one less repo in my build file.

Not sure it helps but I just created https://github.com/facebook/yoga/issues/1093

oprisnik commented 3 years ago

And the Flipper support built-in to react-native since 0.62 depends on Fresco, which is currently available on JCenter only (at least the version RN uses, see facebook/fresco#2585).

We've fixed the Maven release pipeline for Fresco and new versions (starting with the already released 2.5.0) will be on Maven Central.

matinzd commented 3 years ago

Some packages are not available on jcenter and it throws 403 error. I don't know if it is an outage or no.

mikehardy commented 3 years ago

jcenter had an outage today but that's just normal "internet doesn't work all the time because of XYZ" stuff. JCenter has officially said they will maintain service indefinitely, as their latest update. Migrating away from jcenter is still a good thing and react-native in general has already done so as of the template in react-native 0.65 as far as I know. There are only a few packages in widespread use (in my experience, anyway) that still rely on it in their current stable versions and even those are working on migrating as well. This may be closable.

numandev1 commented 3 years ago

as recommended, I can make post-install jcenter to mavenCentral converter like jetifier which will convert all of node_modules jcenter to mavenCentral and will make a ticket for publish on mavenCentral

What you say?

cortinico commented 3 years ago

As a side note, what happened yesterday was an outage of JCenter (a slow response time) which caused a domino effect on Gradle Plugin Portal: https://status.gradle.com/incidents/ndfqcnmqc8n7

Removing jcenter() definitions from the build.gradle files had little effect as fetching of Gradle Plugins was actually causing the failures. There is little to do in those cases, but folks on the Gradle side are looking into it for the future: https://gradle-community.slack.com/archives/CA7UM03V3/p1631786914095300

Currently facebook/react-native is not using jcenter() at all https://github.com/facebook/react-native/search?q=jcenter and I believe we should be fine by closign this issue.

And what about com.facebook.yoga:proguard-annotations? AFAICT react-native directly depends on it (see facebook/yoga#1065 (comment)).

I can see proguard-annotations on Central over here https://search.maven.org/artifact/com.facebook.yoga/proguard-annotations/1.19.0/jar I understand that older versions are missing, but the latest one is available, making possible to build entirely without JCenter

kelset commented 3 years ago

yeah now that 0.65 is out, and that is not using jcenter anymore we can close this one off

raydenvoldeskine commented 2 years ago

I had RN 0.64.x and start getting this error today. I have updated RN to 0.65 as advised in this thread, even removed node_modules before but I still get this message!

Unable to load Maven meta-data from https://jcenter.bintray.com/com/facebook/react/react-native/maven-metadata.xml.

alex-a-pereira commented 2 years ago

@raydenvoldeskine not exactly sure but it looks like bintray went down earlier.. even their status page is 502'ing.

If you try to go to that URL that it's trying to fetch meta-data from it 502's as well

raydenvoldeskine commented 2 years ago

Yes I did that. And I hoped from that thread RN 0.65 solves the issue but the problem is still there for me And also I cannot find any entry of "jcenter" or "bintray" in node_modules, do not currently get where it is coming from.

mikehardy commented 2 years ago

Ugh, looks like bintray is still supposed to be up but I confirm this as well. Appears to be an unexpected outage, separate from their sunset (which was changed to just "sunset on uploading new packages, but we'll keep resolving old ones". Hopefully it comes up soon, react-native-macos currently only goes as high as react-native 0.64 :weary: and I was working on a netinfo macos example

HadiModarres commented 2 years ago

Wasn't RN 0.65 supposed to be not dependent on jcenter anymore? So why does jcenter outage still affect the builds?

mikehardy commented 2 years ago

Good question. Perhaps people did not remove it from their builds? I still have it in mine because I have one stubborn lib that needs it. npx react-native init TestProject followed by cd TestProject && npx react-native run-android will likely work, even if you add --version=0.65.0 to the init line, according to upgrade-helper: https://react-native-community.github.io/upgrade-helper/?from=0.64.2&to=0.65.0

I'm just stuck because work projects have one stubborn lib, and library side-task requires 0.64. No winning here. Hopefully it comes up soon.

raydenvoldeskine commented 2 years ago

I assume in my case the reason was a native module which included jcenter() reference (react-native-zebra-scanner), the module hasnt been updated for three years, so no surprise.

radko93 commented 2 years ago

Expo still includes jcenter in some projects like expo-av or image picker.

laerciokonzen commented 2 years ago

This are my changes on the android/build.gradle to make it work. But it`s taking around 10 times more to get the build done. Any insight?

buildscript {
    ....
    repositories {
        google()
        mavenCentral()
        mavenLocal()
    }
    ....
}
....
allprojects {
    repositories {
        maven { url "https://plugins.gradle.org/m2/" }
        maven { url "https://repo.spring.io/plugins-release/" }
        mavenLocal()
        ....
        ....
        mavenCentral {
            // We don't want to fetch react-native from Maven Central as there are
            // older versions over there.
            content {
                excludeGroup "com.facebook.react"
            }
        }
        ....
        ....
    }
}

mikehardy commented 2 years ago

Standard insight for network related things: You've added a new remote end point that gradle consults for each dependency. That new endpoint must be slow.

bintray is back though, so at least until we're all using good dependencies, we're good again for a while.

If you're using dependencies that are unmaintained and not moved off bintray, fork them, and do the work needed to get them published correctly off jitpack, and you'll be able to move on as well (for example: https://github.com/hieuvp/react-native-fingerprint-scanner/pull/189/files)

cortinico commented 2 years ago

Any insight?

@laerciokonzen You definitely don't need the first two lines. The first is the Gradle Plugin Portal where only Gradle Plugins are hosted, not runtime dependency (hence is not needed). The second is the a Spring Maven Repo, which I also believe is not needed in your specific case (unless you're using some Spring library that is not on Maven Central).

Anyway as the Bintray outage is over, you should be all good by now.

cortinico commented 2 years ago

Wasn't RN 0.65 supposed to be not dependent on jcenter anymore? So why does jcenter outage still affect the builds?

Dropping a couple of clarification point here as there was a lot of confusion in the last days.

That's a link to the incident report on Gradle's end: https://status.gradle.com/incidents/nv93msj8q658

As @HadiModarres asked, RN 0.65 is not dependent on JCenter anymore. However, this outage of Bintray/JCenter was affecting the ecosystem in a number of way:

Expo SDK was based on RN 0.64.x, hence still used jcenter() definition in their build.gradle files
Libraries that you import via autolinking are importing build.gradle files that live inside the node_modules folder inside your build. Should those file contain a jcenter() definition, as maybe the library author hasn't updated them, you'll be hitting JCenter for that specific library.
If you're using Gradle Portal (i.e. you have a plugins{} block, or you have a maven { url "https://plugins.gradle.org/m2/" } or a gradlePluginsPortal() defined in your repositories{} block), you would have not be able to build. This because Gradle Portal is mirroring JCenter. More on this here:
- https://github.com/gradle/gradle/issues/15406

Specifically, @ShikaSD pushed a fix that mitigated this last point yesterday https://github.com/facebook/react-native/commit/3cf0291008dfeed4d967ebb95bdccbe2d52c5b81

That being said, the suggestion is to try to keep your RN version up-to-date as you won't benefit of some of the improvements we pushed if you're missing out on versions.

EDIT: typo

raydenvoldeskine commented 2 years ago

Is it me or Error 502 is happening again right now?

walesonn commented 2 years ago

Yes, It's happen with me too

FrSenpai commented 2 years ago

Yes, It's happen with me too

Same today, sadly ... Need to stop to use jcenter and switch to Maven, but ... meh... so many projects impacted :(

salman1696 commented 2 years ago

adding this to build.gradle works for me. gradlePluginPortal()

emrecanozkok commented 2 years ago

adding this to build.gradle works for me. gradlePluginPortal()

explain please where you added

edit:anyway i added it and my problem seems fixed

edit2: not fixed :)

react-native-community / discussions-and-proposals