Closed hesamdarbandi closed 5 years ago
JonDouglas
commented
5 years ago This was first introduced in Xamarin.Android 9.3. We did add an additional ABI to adhere to Google Play's requirements:
This looks to be a regression with the Linker in which it's preserving more types than it should.
brendanzagaeski
commented
5 years ago msbuild -restore -p:Configuration=Release -t:PackageForAndroid.With Xamarin.Android 9.4.0.51, the resulting APK file is about 11 MiB:
Length Name
------ ----
11334365 com.companyname.androidapp1.apkWith Xamarin.Android 9.3.0.23, the resulting APK file is also about 11 MiB:
Length Name
------ ----
11575307 com.companyname.androidapp1.apkWith Xamarin.Android 9.4.1.0, the resulting APK file is about 18 MiB:
Length Name
------ ----
18837089 com.companyname.androidapp1.apkIt looks like the fix for https://github.com/xamarin/xamarin-android/issues/3263 in https://github.com/xamarin/xamarin-android/commit/81627a4260ac433d58bd2953b7efcca46b89ffb9 is perhaps preserving all of the *Invoker types instead of only those that are actually required for the app.
The outputs from the LinkAssemblies task in obj\Release\90\android\assets are problematic.
Taking the smallest difference in the LinkAssemblies output as an example, the output from ildasm -noil -classlist shows the following differences for obj\Release\90\android\assets\Xamarin.Android.Arch.Lifecycle.LiveData.Core.dll:
// Classes defined in this module:
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
// Class __TypeRegistrations (auto) (ansi)
// Interface IObserver (public) (abstract) (auto) (ansi)
// Class IObserverInvoker (auto) (ansi)
// Class LiveData (public) (abstract) (auto) (ansi)
+// Class ObserverWrapper (abstract) (auto) (ansi) (nested public)
+// Class ObserverWrapperInvoker (auto) (ansi) (nested assembly)
// Class LiveDataInvoker (auto) (ansi)
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~For comparison, the matching file when using Xamarin.Android 9.3.0.23 does not include these 2 classes. Based on that result, it seems these types are not required by the app and that the managed linker is expected to remove them in this scenario
EmilAlipiev
commented
5 years ago Is it not the case for VS mac 8.3 pre version? i see the app size is normal as expected using 8.3pre?
brendanzagaeski
commented
5 years ago Is it not the case for VS mac 8.3 pre version? i see the app size is normal as expected using 8.3pre?
Correct. The change in the linker behavior from the fix for https://github.com/xamarin/xamarin-android/issues/3263 in https://github.com/xamarin/xamarin-android/commit/81627a4260ac433d58bd2953b7efcca46b89ffb9 is not yet present in the current Xamarin.Android SDK version 9.5.0.27 in the Preview channel of Visual Studio 2019 for Mac version 8.3 Preview. That preview version was published on July 9, while the change in the linker behavior was added on July 19. The linker behavior change will be part of the next Xamarin.Android SDK version 9.5 preview, so that next preview will be affected by the size increase.
FANMixco
commented
5 years ago Is it not the case for VS mac 8.3 pre version? i see the app size is normal as expected using 8.3pre?
Correct. The change in the linker behavior from the fix for #3263 in 81627a4 is not yet present in the current Xamarin.Android SDK version 9.5.0.27 in the Preview channel of Visual Studio 2019 for Mac version 8.3 Preview. That preview version was published on July 9, while the change in the linker behavior was added on July 19. The linker behavior change will be part of the next Xamarin.Android SDK version 9.5 preview, so that next preview will be affected by the size increase.
Hi @brendanzagaeski , therefore, the new apps will be bigger? Or there will any solution?
brendanzagaeski
commented
5 years ago Based on my investigation so far, this issue #3454 is not expected behavior. The next Visual Studio 2019 version 16.3 Preview will include this bug. But I would expect for the issue to be resolved in a future Visual Studio 2019 version 16.3 Preview (and quite possibly a future Visual Studio 2019 version 16.2.x release as well).
Magendanz
commented
5 years ago Is there any workaround, so that we're not all stuck on 16.2.0 in the interim. Is there even a way to easily roll back the 16.2.1 update? And if we're forced to do a clean install, will VS Installer even let us install an older build?
brendanzagaeski
commented
5 years ago The team members who work on the managed linker can update about a workaround if I'm mistaken, but I am not aware of one so far. (As a side note, the extra types included in the APK shouldn't cause any other behavior problems when running the APK on device, so these larger APKs would be OK to distribute in cases where the extra size itself isn't a concern for the application's user base.)
Previous versions of Visual Studio 2019 Enterprise and Visual Studio 2019 Professional are available on https://docs.microsoft.com/visualstudio/releases/2019/history. Those installers do require uninstalling the previously installed version.
Previous versions are not available for Visual Studio 2019 Community. The links for the Xamarin.Android SDK version 9.4.0.51 package are available for Windows and macOS, but replacing just the Xamarin.Android SDK package is not a standard scenario. See https://github.com/xamarin/xamarin-android/issues/3112#issuecomment-495780758 for a few more details about that approach.
ElteHupkes
commented
5 years ago Phew I'm glad I found this issue. Our app size increased from 25mb to 32mb overnight, after I enabled and tested everything with d8 and r8... Once I realized that the app size on the current master was also a lot larger I came here, and there you go.
They're not earth shattering size increases but it is kind of annoying. Any time frame on when this might be addressed?
tranb3r
commented
5 years ago Once again, this raises a series of questions:
Let's hope the VS and Xamarin teams will take this issue seriously, release a fix immediately, and take this opportunity to improve their processes at the same time...
cc @pierceboggan cc Jacqueline Widdis (are you on github?) cc Maddy Ledger (are you on github?)
EmilAlipiev
commented
5 years ago @tranb3r it can't be put in words better way. There is also this thing, Xamarin team keeps sending us surveys regarding "what can we do better to increase productivity". Answer is keep VS and Xamarin.Android updates stable or if not, react on vital problems faster. This bug costed me a lot of time. Because i initially though, i messed up something in my code or 3rd party nuget updates. i was comparing my branches, commits in recent days, uninstalling 3rd party nuget updates and finally found out that it was caused by VS update. how awkward is that :/ I have sent a personal email to @davidortinau with all VS2019 and VS4Mac issues with links. it is just unbearable if your IDE isn't working as expected and no easy way to go back to previous. I don't really care if any nuget update is broken, i can live with the older version but not when IDE.
MitchBomcanhao
commented
5 years ago The exact same scenario has occured a few weeks ago, when AOT was broken for xamarin.android in one of the VS preview release. No mention in the release notes, no workaround, no way to rollback, and it has taken several weeks to get a fix. Such a waste of time.... and lesson not learned.
This reminds me of the Profiled AOT for android, which was broken on windows on the first preview release. It seems like they use Macs quite a lot and release stuff that works on Mac but not on windows, and don't even mention that it is broken in the release notes. "hey here's this new thing", then you spend a chunk of time trying it out and it just doesn't work, only to find that it is fundamentally broken and they knew about it in advance (by reading the issues in other projects).
JonDouglas
commented
5 years ago Hey everyone,
We apologize profusely for this regression being introduced into Visual Studio 16.2. While the backing fix that introduced this behavior was intended to solve a number of crashing issues that had been previously reported, we sadly did not notice the consequences in which inflated package size as described by @brendanzagaeski in previous comments.
We are investigating alternative approaches, and will be working diligently to include a fix in a future service release and preview release of Visual Studio while ensuring this does not happen again. We will keep you updated as progress is made, and we are extremely grateful for your patience.
We appreciate all of the feedback regarding concerns with quality and I want to assure you that this is at the forefront of our minds when we ship updates to the product.
If there is any other feedback, comments, or concerns that you would like to discuss, please do not hesitate to email me at jodou @ microsoft.com.
EmilAlipiev
commented
5 years ago @JonDouglas thank you for the honest words. done is done. we should learn from it. I think that it would be nice to push such bug fixes into pre first where people can test and deliver feedback because this bug was affecting very little amount of people but consequences effected almost everyone. Now my question is i see that there is new 8.3 pre 2 update for Mac. is that Android size bug included? in the release notes i dont see anything in "known issues". Because 8.3 pre 1 doesnt have that bug and i would like to stay on this if pre 2 includes the bug.
tranb3r
commented
5 years ago @jondouglas Thanks for your comment, but I do not have the feeling that you answered my questions. If "quality is at the forefront of your minds" as you've said, why did you release a preview of 16.3 yesterday, knowing that it would include the same regression that was reported a few days ago in 16.2 ?
brendanzagaeski
commented
5 years ago Now my question is i see that there is new 8.3 pre 2 update for Mac. is that Android size bug included? in the release notes i dont see anything in "known issues".
At the moment, the known issues for Xamarin.Android are distributed across the Visual Studio release notes and the Xamarin.Android SDK release notes. For this particular case, the known issue in the Xamarin.Android SDK version 10.0 preview can be found in the Xamarin.Android SDK release notes on docs.microsoft.com and GitHub. I have some ideas for things to try to make the list of known issues for the Xamarin.Android SDK easier to find and browse in the future.
brendanzagaeski
commented
5 years ago why did you release a preview of 16.3 yesterday, knowing that it would include the same regression that was reported a few days ago in 16.2 ?
For the question about why the fix for https://github.com/xamarin/xamarin-android/issues/3263 was included in yesterday's Visual Studio 2019 version 16.3 Preview release, another team member can correct me if I'm mistaken, but I think for this specific scenario where a problem is introduced by a fix in a servicing update in the Release channel rather than in a minor update (terms borrowed from the release rhythm document), the fix will always also have been committed to the master and current Preview branches to maintain branch consistency (for both better and worse). In this particular case, in addition to the branch consistency, there would potentially also be trade-offs to weigh between addressing the issue with missing *Invoker types (which could cause apps to crash at an unexpected point, potentially on a relatively rarely used code path) versus increasing the Release configuration APK size by several megabytes (which is perhaps easier to notice than a crash on certain code paths). That trade-off would need to be weighed in the Preview version context, taking into account that the version is not yet "go-live" and not yet intended for use in production code (terms borrowed from the Visual Studio release notes).
FANMixco
commented
5 years ago If "quality is at the forefront of your minds" as you've said, why did you release a preview of 16.3 yesterday, knowing that it would include the same regression that was reported a few days ago in 16.2 ?
Another team member can correct me if I'm mistaken, but I think for this specific scenario where it's a servicing update in the Release channel rather than a minor update (terms borrowed from the release rhythm document), the fix will always also have been committed to the
masterand current preview branches to maintain branch consistency (for both better and worse). In this particular case, in addition to the branch consistency, there would potentially also be trade-offs to weigh between addressing the issue with missing *Invoker types (which could cause apps to crash at an unexpected point, potentially on a relatively rarely used code path) versus increasing the Release configuration APK size by several megabytes (which is perhaps easier to notice than a crash on certain code paths). That trade-off would need to be weighed in the Preview version context, taking into account that the version is not yet "go-live" and not yet intended for use in production code (terms borrowed from the Visual Studio release notes).
Hi @brendanzagaeski, the only issue that I see is that since a couple of releases ago in VS2019 as I emailed exchange with some people from Microsoft is that Xamarin.Android has become quite unstable.
I got a couple of emails asking me for providing my feedback about my experience with VS 2019 for Windows and Mac (because I have both) and I got in contact with some people from MS as I said before where they asked me to provide the full list of bugs that I have reported everywhere (GitHub, VS Developer Community and Xamarin Forums).
I've been using Xamarin.Android with VS2015, 2017 and currently 2019 and I am somehow astonished of the current situation. This is part of one of my emails:
"At this point of time, I'm having several mixed feelings about Xamarin.Android because it's quite unstable as a solution since I cannot release anything properly to production because each new release I should wait and see what the next collection of bugs is."
I can understand it's not prepared for production, but I'd say our concern is: When will we have a proper solution for production? I'm sure that even at my work where I have the Professional Version of 2019 installed, I would have a collection of bugs in the stable branch that it will be difficult to mitigate and justify many of them and to go backwards, I will need to start testing and uninstalling until I find, which version of VS is somehow working.
Also, I understand that is not easy to test a complex solution as Xamarin, there is a complex CI/CD, Unit Tests and probably a couple of integration tests, but I feel that something is missing since VS2019 was released.
brendanzagaeski
commented
5 years ago Just to clarify, my previous comment is only about the precise question of why the fix for https://github.com/xamarin/xamarin-android/issues/3263 is included in Visual Studio 2019 version 16.3 Preview 2. I've adjusted the wording of that comment to better reflect that.
FANMixco
commented
5 years ago Just to clarify, my previous comment is only about the precise question of why the fix for #3263 is included in Visual Studio 2019 version 16.3 Preview 2. I've adjusted the wording of that comment to better reflect that.
Hi @brendanzagaeski , you can take my case as an example since the #3263 is one of so many that I reported, but I repeat my words:
"At this point of time, I'm having several mixed feelings about Xamarin.Android because it's quite unstable as a solution since I cannot release anything properly to production because each new release I should wait and see what the next collection of bugs is."
And this is why I had email exchanges with MS people because as I said, I could go to my work and use the Professional Edition (not Community) where I have the benefit to rollback to a previous edition, but I will need to start uninstalling a massive application (specially in Windows) then install in it again and start testing for hours until I found one where is working properly. How many hours or days I need to spend for that? Maybe 2 or 3h per test. As in your case and my case, I don't think my manager would like to know that I'm stopping my entire work of a day or days because I need to reinstall each version of VS until I find which one works and which one doesn't.
Keep in mind that I gave the previous example to highlight the problem of the stability of the tool, but I'm sure something is missing in the QA section, either is the pressure of Android Q and Xamarin.Android 10 or something else, but I don't like the justification that you said that this is not a release to use in production because in my experience Stable ones as Previews are extremely unstable are alike at this point of time, when I reported the first bug it was in a Stable branch and then in the preview branch and then, I brought another bug that was in the stable, I use stable branches in Mac and Previews in Windows.
Someone asked about a possible timeline for the solution and this could be at least some light of where is going, I'm not sure the root cause of the current situation, but probably some tests are missing.
brendanzagaeski
commented
5 years ago Thanks for the feedback! I wasn't intending for my comment to suggest that a pattern of problematic issues in new releases is acceptable at all. I wouldn't be able to help answer bigger questions about a pattern of several different problematic issues across multiple releases. I just wanted to offer an explanation for the particular case of why this one problem would be added to the Preview version when it was already in the Release version in case anyone might have been curious about that. Apologies if it sounded like I was suggesting something more than that!
FANMixco
commented
5 years ago Thanks for the feedback! I wasn't intending for my comment to suggest that a pattern of problematic issues in new releases is acceptable at all. I wouldn't be able to help answer bigger questions about a pattern of several different problematic issues across multiple releases. I just wanted to offer an explanation for the particular case of why this one problem would be added to the Preview version when it was already in the Release version in case anyone might have been curious about that. Apologies if it sounded like I was suggesting something more than that!
Hopefully @brendanzagaeski the solutions will come soon, since I already reported (15-Aug-2019 10:30 AM) a new bug in the "Stable" VS 2019 for Mac because the custom controls are not being recognized anymore by the IntelliSense, but they can still compile and work. I know this is Community Edition because I'm not at work today, but I'm sure if I try in my work, it will be similar in the Professional or maybe another one will be there.
radekdoulik
commented
5 years ago https://github.com/xamarin/xamarin-android/pull/3525 should improve the apk size and still fix the original crash.
akamud
commented
5 years ago I read the issue and I can assume that the referenced Linker commit makes it unfeasible to go back to the same size as before this fix. Is this correct?
tranb3r
commented
5 years ago @xamarin-release-manager @JonDouglas Can we expect a fix to be released soon ?
brendanzagaeski
commented
5 years ago A new Preview version has now been published that includes a fix for this item. A new Release version is also planned that will include the fix. I will update this issue again when the Release version is available.
Fix included in Xamarin.Android 10.0.0.40.
Fix included on Windows in Visual Studio 2019 version 16.3 Preview 3. To try the Preview version that includes the fix, check for the latest updates in Visual Studio Preview.
Fix included on macOS in Visual Studio 2019 for Mac version 8.3 Preview 3. To try the Preview version that includes the fix, check for the latest updates on the Preview updater channel.
akamud
commented
5 years ago With this fix we should see the app size exactly the same as before?
brendanzagaeski
commented
5 years ago Radek can correct me if I'm mistaken, but as I understand it, the fix isn't a pure reversion of the change that introduced the size increase. It is instead a new companion fix that allows the linker to remove the unneeded *Invoker types, bringing the app sizes down much closer to where they were before.
The commit that brings in the fix includes the following table in the commit message that shows the results for one example app:
| Version | Size in bytes |
|---|---|
| Before @4d8c28 | 21434941 |
| After @4d8c28 | 28242405 |
| This fix | 21638678 |
So in that example, the result after this fix is about 200 kilobytes larger than the Before row, but down from 7 megabytes larger in the After row.
tranb3r
commented
5 years ago With the fix, my app is still 10% bigger than before. Of course, it's better than +25% without the fix...
radekdoulik
commented
5 years ago It is as Brendan describes, we still need to keep some of the *Invoker types to avoid crashes in some of the calls from java side.
The crashes were result of linker regression in the past, we were keeping these *Invoker types before the regression as well.
radekdoulik
commented
5 years ago @tranb3r 10% increase seems a bit larger than expected. In the sample I tested it was around 1%.
What are the apk sizes you see? Which assemblies do you reference and how many of them contain java bindings?
tranb3r
commented
5 years ago Apk size:
I'm using AOT+LLVM, and the average increase of .so files seems to be around 10.5%. But interestingly, the increase of libmonodroid_bundle_app.so (bundle of the dlls) is only about 2.5%. So maybe there is another issue specific to aot/llvm ?
About java bindings, I'm using a lot of nugets, and I have no idea how many of them contain java bindings. But pretty much every single .so file is increasing in size in the apk.
brendanzagaeski
commented
5 years ago Release status update for the initial candidate fix that was published in a Preview version yesterday
(This candidate fix does not yet involve any new investigation on the results discussed yesterday and today about the AOT+LLVM scenario.)
A new Release version has now been published on Windows that includes the initial candidate fix for this item.
Fix included in Xamarin.Android 9.4.1.1.
Fix included on Windows in Visual Studio 2019 version 16.2.4. To get the new version that includes the fix, check for the latest updates or install the latest version from https://visualstudio.microsoft.com/downloads/.
By a chance of the release cadence match-up, the corresponding update for Visual Studio 2019 for Mac is still upcoming. I will update this issue again when that version is available.
hanseul1795
commented
5 years ago @brendanzagaeski is there an azure link for the 10.0.0.40 version with this fix?
I'm using a custom build system and need to download the .zip or the .vsix file
tranb3r
commented
5 years ago @brendanzagaeski @radekdoulik @JonDouglas Any update on the aot/llvm issue ?
brendanzagaeski
commented
5 years ago is there an azure link for the 10.0.0.40 version with this fix?
@hanseul1795, sure thing. If you need it, you can download the latest .vsix installer directly: Xamarin.Android.Sdk-10.0.0.40.vsix
tranb3r
commented
5 years ago @brendanzagaeski @radekdoulik @JonDouglas Any update on the aot/llvm issue ?
As of now, I still haven't received any answer to my questions. I don't even understand why this issue has been closed. I would really appreciate if you could take 1 min of your time to post an update. Thanks.
jorgenstorlie
commented
5 years ago It should not be closed. Only workaround is to downgrade.
I used 1 day to figure out the reason my app increased in size. Read this: https://devblogs.microsoft.com/xamarin/shrinking-android-app-size/
What is the reason to optimize your app then Xamarin, messes up everything?
radekdoulik
commented
5 years ago @tranb3r 2.5% increase with lots of java bindings looks right. Let see what makes the rest of the difference. Could you check the apk content?
The apk is a zip archive, could you get us the lists of both apk's, like output of unzip -l your-Signed.apk?
MitchBomcanhao
commented
5 years ago from my end, the current android build has gone back to the same size as it was before the issue was introduced, so it seems resolved, but i'm not using aot at all.
tranb3r
commented
5 years ago @tranb3r 2.5% increase with lots of java bindings looks right. Let see what makes the rest of the difference. Could you check the apk content?
The apk is a zip archive, could you get us the lists of both apk's, like output of
unzip -l your-Signed.apk?
+2.5% is for dlls. But more than +10% for .so libs (when using aot+llvm) !! I'll provide the details you ask later, but again, it's such a waste of time...
radekdoulik
commented
5 years ago @jorgenstorlie the reason for the increased size is that linker became too aggressive at some point and removed too much. We fixed that and so it it is larger again, so we went back to the previous sizes.
Please make sure to use Xamarin.Android.Sdk 10.0.0.40 or newer as there was a regression introduced when we first fixed the issue (in earlier version).
radekdoulik
commented
5 years ago @tranb3r I will try to compare our sample sizes with AOT/llvm using version before the linker breakage and current version to see how it compares.
radekdoulik
commented
5 years ago @MitchBomcanhao thanks for the confirmation
jorgenstorlie
commented
5 years ago I must wait for a stable release 1) I use Appcenter to build my app. 2)I will not use a preview version for release.
The latest "stable" version has this bug.
Do you have release date for a stable release?
tranb3r
commented
5 years ago @radekdoulik
Details for .so libs (16.3.0-pre1 vs 16.3.0-pre3) :
EmilAlipiev
commented
5 years ago Which version of VS did you use to get this difference?
tranb3r
commented
5 years ago Which version of VS did you use to get this difference?
As mentionned previously:
EmilAlipiev
commented
5 years ago I tried on VS Mac 8.3 preview (latest)- my app size is 28mb using VS Mac 8.1 pre- 25 mb, VS Mac 8.2 pre 32mb. yes there is an improvement on VS Mac but I dont know why there is this 3 mb more still. It isn't comprehensive for me why we should still preserve linker to remove this things. Because I never had any problem so far regarding what was reported? Does that problem really target all Xamarin apps or for some specific use cases? if it is some specific use cases, can there be a workaround using "skip" or "preserve" instead of preserving them for all users. we fight for every byte of app size and even 3 mb is something for me and it disturbs me to be honest
tranb3r
commented
5 years ago ~~More information about my app: I'm referencing Xamarin.Forms.DebugRainbows for debugging purpose, but in this version of the apk, I'm not using this dll (the code using it is commented). So I'm expecting the linker to totally remove this dll, which was the case before the regression, but it's not the case anymore. To me, it's one more proof that something is obviously wrong with the linker now.~~
after upgrade vs for mac ,release apk of new blank android project increase to 17 mb