Kartik-715 commented 3 years ago

With the release of iPadOS and iOS 15, do you have plans to migrate this extension to a safari web extension? Could be pretty useful!

quoid commented 3 years ago

@Kartik-715

I have to read more about the support. It seems like Apple is supporting Safari WebExtensions in iOS, not Safari App Extensions. This extension is an App Extension not a Web Extension.

If that is the case the extension would need to be re-written and current functionality could be lost. At the current time I do not know.

Kartik-715 commented 3 years ago

Ahh I see! Anyway I'm going to try and migrate the extension from app extension to web extension! Let's see if that's possible!

quoid commented 3 years ago

@Kartik-715 It is possible it's just a matter of time and rewriting certain functionality like interacting directly with the file system. I don't have any doubts it is possible.

I am waiting until more information is out about the new iOS 15 features before I make any plans. However, once I hear more I will likely be re-writing, so I would suggest not to waste your time attempting to migrate anything. If you want to help when I begin the rewrite that would be more beneficial to the project.

Reinachan commented 3 years ago

You implement Safari web extensions as macOS or iOS app extensions to provide a safe and secure distribution and usage model. You can distribute a Safari web extension with a Mac app, an iOS app, or a Mac app created using Mac Catalyst. You must use Xcode to package your extensions for testing in Safari, and you must be a member of the Apple Developer Program to distribute them through the App Store. For more information, see Apple Developer Program. https://developer.apple.com/documentation/safariservices/safari_web_extensions

Looks like Safari App Extensions and Safari Web Extensions will work on iOS?

quoid commented 3 years ago

@Reinachan

Looks like Safari App Extensions and Safari Web Extensions will work on iOS?

I read the quote, but can you elaborate on why you think Safari App extensions will work? Specifically app extensions without WebExtension components.

Reinachan commented 3 years ago

Ah, without web extension components. Well, then I wouldn't know. I added a questionmark to it as the wording by Apple was a bit vague.

Looking deeper into the various documentation pages, it looks like it's Safari Web Extensions but with the ability to have a native app to control it from.

It also looks to me like they're moving away from Safari App Extensions in general and toward Web Extensions instead. Might be worth considering refactoring to Web Extension either way, as they might just end up cutting support for Safari App Extensions down the line. Who knows.

Moving to Web Extensions could also have the advantage of making it much easier to enable using the extension in other browsers too from what I can gather?

quoid commented 3 years ago

@Reinachan

Thanks for the clarification. I agree that migrating to Web Extension will prove valuable and that is my goal, however I can not give a firm estimate on when that will be complete as it is will take some time.

That being said, simply converting to being predominately WebExtension API won't instantly enable usage in iOS as there are other aspects of the extension to consider, such as selecting the "scripts folder" and mobile considerations for the extension page (ie. editor which will likely barely function on any mobile device currently).

It also looks to me like they're moving away from Safari App Extensions in general and toward Web Extensions instead. Might be worth considering refactoring to Web Extension either way, as they might just end up cutting support for Safari App Extensions down the line. Who knows.

Once they announced WebExtensions, I had the same thought. It's already sort of happening since the WebExtension API is much more robust compared to the App Extension API (which gets seldom updates). It does slightly irk me that Apple decided to drop Safari JS Extension API and force users into App Extension API only to (a few years later) seemingly begin abandoning App Extension API in favor of WebExtension API. It's better for the browser but reveals their finicky and haphazard forward thinking (in terms of Safari).

Moving to Web Extensions could also have the advantage of making it much easier to enable using the extension in other browsers too from what I can gather?

Very true, however at the current moment I don't have much interest in adding additional browser support since there are really good options for those browsers. But that opinion could change in time.

Anyway, WebExtension refactor is planned, stay tuned!

Reinachan commented 3 years ago

You've got some good points in regards to an iOS port. I think that for an initial release, simply giving the option to add/remove userscripts/styles you find on the web could be enough, and potentially expand upon it later.

Personally, I just want to use the userstyles I write on my PC on iOS for instance, and I will assume that a majority of people who use plugins like this don't write them themselves.

I don't know what APIs they give you access to, but considering they let you add a native app to control certain stuff in the extension, perhaps you also get access to a folder in the filesystem which you can store things in? In that case, you might not even need some way to write them, as users could just write them in their preferred program and add it to that folder?

I'd contribute to this but my Macbook decided that it no longer wants to boot regularly (only if in safemode), so I had to install Linux on it to even be able to continue using it. Means I won't really be able to test anything or deploy it to my iPad :/

quoid commented 3 years ago

I don't know what APIs they give you access to, but considering they let you add a native app to control certain stuff in the extension, perhaps you also get access to a folder in the filesystem which you can store things in? In that case, you might not even need some way to write them, as users could just write them in their preferred program and add it to that folder?

I think that would exactly what I'd want to see in a v1 for iOS - simply the ability to read userscripts from a directory of the user's choosing. For example, I store all my scripts in an iCloud folder, if I am able to read from that directory and utilize the userscripts located there, on my phone, I would be very pleased.

Long term possibilities are endless, but I am fairly certain that creating an editor for iOS or iPadOS is more than likely not a goal.

I'd contribute to this but my Macbook decided that it no longer wants to boot r

You already are contributing by sparking up conversation and it's valuable!

quoid commented 3 years ago

track progress here:

145

https://github.com/quoid/userscripts/milestone/6

https://github.com/quoid/userscripts/tree/v4.0.0

quoid commented 3 years ago

This is still being investigated. I am unsure how to go about configuring the project to support iOS. Apple suggests using a converter to add support for other platforms, but unfortunately the converter does not seem to run properly on my computer.

For example, running:

xcrun safari-web-extension-converter --rebuild-project /path/to/project.xcodeproj

Doesn't do anything but display the help text for the converter.

If anyone knows how to add support please send me a message. My email is on my profile page.

julianpomper commented 3 years ago

@quoid I think I found your issue. The problem is that the --rebuild-project option is just available with the Xcode 13 developer tools. So if you run xcrun but did not change the active development directory to the Xcode 13 tools it does not find this option.

You can try entering sudo xcode-select -s path/to/Xcode.app (path to Xcode 13, you can change it back later). Now the command xcrun safari-web-extension-converter --rebuild-project /path/to/project.xcodeproj should work 👍

quoid commented 3 years ago

@julianpomper I had tried that in the past and just tried it again this morning and the results are the same. The converter does not do anything but print out help text. Interestingly the help text does not state that --rebuild-project is a valid option.

Have you successfully rebuilt an existing macIS webextension? If so, what OS are you on and Safari version? I am using Safari 14.1.1 and macOS 11.4 and just figured the tool wasn't available fully until the next macOS.

quoid commented 3 years ago

@julianpomper

option is just available with the Xcode 13

I re-read your post and realized I am on Xcode 12... Sorry it is early for me 🙂

I reckon I won't be able to covert and build the iOS side until I am on the new macOS (or download Xcode Beta). When that gets released I should be able to update and run the convertor and start that process.

julianpomper commented 3 years ago

Exactly you should already be able to do that now with macOS 11.4. Download the Xcode 13 RC here in the Beta tab. After that open it once and install the additional tools. If that is done you should be able to do the steps I described above:

You can try entering sudo xcode-select -s path/to/Xcode.app (path to Xcode 13, you can change it back later). Now the command xcrun safari-web-extension-converter --rebuild-project /path/to/project.xcodeproj should work

Edit: and of course open the Xcode project with Xcode 13 from now on 😄 Edit2: I think this can be also helpful: https://developer.apple.com/documentation/safariservices/safari_web_extensions/optimizing_your_web_extension_for_safari

quoid commented 3 years ago

@julianpomper I just wanted to give you an update. The good news is that I was able to convert the extension to iOS 🎉

The bad news is that a good deal of the v4.0.0 code needs to be re-written #if os(iOS). This will likely take a long time unless I can find an experienced iOS developer to help with this.

EthanG45 commented 2 years ago

What do you mean rewritten? Coming in here with no context just interested in seeing any user script app reach iOS. Do you mean the scripting code no longer works or just UI is broken?

quoid commented 2 years ago

What do you mean rewritten?

@EthanG45

Here's some examples:

Currently the app uses security scoped bookmarks, which are set by the user through the containing app, to set the read/write location. The way this is written in macOS is not comptatible with iOS.
By default the app reads from the users .documentsDirectory, but for the iOS version there'd have to be a specific workflow which compels the user to set a shared folder as a read location. Creating userscripts is not in the scope for the iOS version.
When converting the extension is does not keep any of the swift functionality that is included in the current and beta versions. That needs to be integrated back in after the conversion, or better yet, the conversion needs to happen, then that converted version needs to be integrated into the current project (or some similar approach).

Anyway, my point is there's a lot of stuff that needs to get to done to even figure out if it is viable to run on iOS and iPadOS (I think it will). Some of that includes iOS development which I am inexperienced with. If all of this is left up to me, it will probably get done, but just take quite a while.

Do you mean the scripting code no longer works or just UI is broken?

I reckon it's mostly UI but some of the scripting code needs to be adapted. As mentioned above about the file read location - that is at the core of how userscript injection works. But most of the injection core happens on the WebExtension side which seems supported according to the documentation.

Also note that the iOS parts seemingly can co-exist with the macOS. So it's really more about created some iOS specific code rather than a "rewrite".

EthanG45 commented 2 years ago

Ah I see that is a lot of non-easily converted code. I've got some naive iOS experience mostly on the UI side, so I can't say I could contribute much, at least right now I've got a full plate. Hopefully, one day userscripts can come to iOS still.

quoid commented 2 years ago

Hopefully, one day userscripts can come to iOS still.

It might take a little bit but I think it will get there @EthanG45

quoid commented 2 years ago

Update

I've been running some tests. Outside of adapting current code to work on iOS, I am still in the process of figuring out if choosing a read location can persist between Safari launches. Choosing a location for reading the userscripts will have to be done as it is currently done in the current macOS app. That means the user selects this location in the containing app, in this case the iOS app and then a sharedbookmark is created and stored in a shared userdefaults file.

That shared userdefaults value is then read by the extension. In the simulator, it seems like this persists, but I am unsure how this will function on an actual phone/ipad. I've read some random forum posts suggesting that it is not possible to have sharedbookmarks persist as they can in macOS. If that is true, I am not sure how to approach all of this. However, it seems to persist currently, so I am hoping it is not true.

Once that is figured out, I think the path is clear. I'm successfully running this in the simulator currently.

EthanG45 commented 2 years ago

@quoid That's fantastic to hear! Looking forward to future updates on development progress.

quoid commented 2 years ago

Update

I updated my iPhone 8 to ios 15 and loaded the extension. Injection seems to occur, will update everyone on if the bookmark persists. However, the popup does not load properly. This is not something I experienced in the simulator.

I made a thread in the Apple forums in hopes of deciphering the error code: https://developer.apple.com/forums/thread/691389

If anyone has any insight please post here

quoid commented 2 years ago

Update

The iOS app has been on my phone for a bit over a day and is still working. The popup still randomly fails, but subsequent reloads after a failure seemingly resolves it. I can't figure out what's causing it to fail so I opened a dev support ticket with Apple.

The iOS app ui is terribly bland at the moment, but before putting resources into prettifying it, I want to validate functionality.

I will be looking into setting up Test Flight this weekend so I can get more hands on this to help out with testing.

cobaltt7 commented 2 years ago

Thank you for your work in maintaining this extention (and especially making it free :P)

From: Justin Wasack @.> Sent: Monday, October 4, 2021 6:36:54 PM To: quoid/userscripts @.> Cc: Paul Reid @.>; Manual @.> Subject: Re: [quoid/userscripts] iOS + iPadOS Support (#135)

Update

The iOS app has been on my phone for a bit over a day and is still working. The popup still randomly fails, but subsequent reloads after a failure seemingly resolves it. I can't figure out what's causing it to fail so I opened a dev support ticket with Apple.

I will be looking into setting up Test Flight this weekend so I can get more hands on this to help out with testing.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/quoid/userscripts/issues/135#issuecomment-933935707, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOT5DESEMPPJZUMYSTIN67LUFI3BNANCNFSM46JMRWLQ.

EthanG45 commented 2 years ago

Awesome! Definitely share the TestFlight as I can put it through its paces with some of my scripts.

NewsGuyTor commented 2 years ago

Update

The iOS app has been on my phone for a bit over a day and is still working. The popup still randomly fails, but subsequent reloads after a failure seemingly resolves it. I can't figure out what's causing it to fail so I opened a dev support ticket with Apple.

Just throwing it out there, but have you tried the iOS 15.1 beta? Might be an already fixed OS/Safari issue causing it.

quoid commented 2 years ago

Just throwing it out there, but have you tried the iOS 15.1 beta? Might be an already fixed OS/Safari issue causing it.

Thanks for the suggestion @NewsGuyTor, I haven't installed it, but that would be great if it did. Hopefully I can get this Testflight stuff figured out soon so I see if other people experience it as well.

quoid commented 2 years ago

I just deleted a comment that was promoting a paid app that someone was promoting.

I don't think promoting paid apps is relevant to this issue. If it were discussing another open source project that could help with the development of this feature the comment would remained.

a9eelsh commented 2 years ago

I didn't promote it, so I meant that someone supported userscript, that is, you can do it and I gave you an example of that.

quoid commented 2 years ago

@a9eelsh

I didn't promote it, so I meant that someone supported userscript, that is, you can do it and I gave you an example of that.

Linking to a paid closed-source application, kind of the antithesis of this project, is promoting it.

This issue thread is a focused discussion about iOS and iPadOS support for this extension. There are plenty of open ended discussions going on elsewhere, like Reddit, to talk about and promote products you like. Frankly, it's not been easy work trying to get iOS support for this extension and to have someone who hasn't contributed to this conversation (or even seemingly uses this extension) drop into an issue thread just to promote a paid product that seemingly includes some of the functionality I've been trying to implement is deflating.

It's not just the comment, but it is how you commented. Something about the fact that is was not in direct response to anyone in particular and the nature of where it ended up makes it seem opportunistic rather than an attempt to actually solve any problem

Anyway, I will quote what you wrote and let others decide:

Hello can use https://apps.apple.com/sa/app/hyperweb/id1581824571 support userscript

madman404 commented 2 years ago

I feel at the mention of "that app", I should further emphasize how much that isn't helpful by emphasizing the reason open-source projects like this are good for the community to begin with. I'd like to use userscripts without as much device information as can be gathered being collected and tied to my unique IP address as possible.

willl commented 2 years ago

I'd like to use userscripts without as much device information as can be gathered being collected and tied to my unique IP address as possible.

It also seemed to do a bit of "everything" rather than an extension that allowed the user to make use of userscripts.

Anyway... Back on topic, I look forward to testing this extension whenever it is made available.

quoid commented 2 years ago

I'd like to use userscripts without as much device information as can be gathered being collected and tied to my unique IP address as possible.

I admit I did some digging into that app after it was posted. I took a look at their privacy policy and while their private information gathering isn't as egregious as some other companies, it's more than I want to give up to manage some userscripts. I always find it ironic when companies make products that are supposed to protect users (ie. adblockers or purporting a "100% private experience") are themselves gathering private information from their users. And also the $2.99 monthly cost, that's pretty steep as well. However, I admit the app seems to do a lot and I am sure many people find it useful.

That all being said, I submitted a build to Testflight tonight. I hope it gets reviewed by tomorrow.

I have no idea how the app will function on other people's devices so I would assume the first build to be more an alpha than a beta. On my iphone 8, I've been running the extension for about a week and it still functions as I expect (with an intermittent popup bug mentioned above) - however I have not done pushed it very hard in that time. The big test will be if other people can install it, read from a location selected from the Files.app (ie. icloud folder); I am eager to find that out.

Edit 2021/10/10: still pending approval ¯_(ツ)_/¯

a9eelsh commented 2 years ago

I'd like to use userscripts without as much device information as can be gathered being collected and tied to my unique IP address as possible.

I admit I did some digging into that app after it was posted. I took a look at their privacy policy and while their private information gathering isn't as egregious as some other companies, it's more than I want to give up to manage some userscripts. I always find it ironic when companies make products that are supposed to protect users (ie. adblockers or purporting a "100% private experience") are themselves gathering private information from their users. And also the $2.99 monthly cost, that's pretty steep as well. However, I admit the app seems to do a lot and I am sure many people find it useful.

That all being said, I submitted a build to Testflight tonight. I hope it gets reviewed by tomorrow.

I have no idea how the app will function on other people's devices so I would assume the first build to be more an alpha than a beta. On my iphone 8, I've been running the extension for about a week and it still functions as I expect (with an intermittent popup bug mentioned above) - however I have not done pushed it very hard in that time. The big test will be if other people can install it, read from a location selected from the Files.app (ie. icloud folder); I am eager to find that out.

I’m waiting for you 😍

a9eelsh commented 2 years ago

Hey bro, any news?

quoid commented 2 years ago

Unfortunately it's still in review. It's taking and incredibly long time. I don't know if this is normal for iOS apps but on macOS it usually only takes a day.

It's been about 6 days and it's just a TestFlight build, but I guess those go through the same review process.

a9eelsh commented 2 years ago

Can you give existing here IPA for quick experience?

quoid commented 2 years ago

@a9eelsh

Can you give existing here IPA for quick experience?

I am afraid I don't know what that is. I'm not an iOS developer so this is all new to me.

quoid commented 2 years ago

@a9eelsh

I deleted the duplicate comment.

https://wiki.genexus.com/commwiki/servlet/wiki?34616,HowTo%3A+Create+an+.ipa+file+from+XCode Example

I see now. I don't have time right now to do that, but if this takes very long I will consider it.

quoid commented 2 years ago

Today I got a message from Apple Review, they said they needed more information before they could review. I had to explain what the app/extension does and provide some sample userscripts. Hopefully it satisfies their request and it gets approved.

quoid commented 2 years ago

@Kartik-715 @willl @madman404 @Reinachan @julianpomper @EthanG45 @RedGuy12 @a9eelsh and anyone else viewing this thread. Version 0.1.0 beta for iOS got approved for testflight.

If you want to test, please sign up here: https://forms.gle/QB46uYQHVyCxULue9 - don't forget to use the email associated with your Testflight account. Even if you signed up in the past to beta test, you need to re-sign up because I just purged the old tester pool, to prevent sending unwanted Testflight invites.

I reckon this first version will be rough, so please adjust expectations. At the current time I am mainly looking for iPhone testers running iOS 15+. The iPad version doesn't really work at the moment (visual bugs with the popover).

bradenglund commented 2 years ago

I was able to install and enable as expected. Unfortunately I can’t get past the “Something went wrong - click to retry” message despite force closing safari, app, and disabling/re-enabling extension.

I have the userscripts folder saved on iPhone, not iCloud if that matters.

quoid commented 2 years ago

@bradenglund

That is the issue I commented on above. I was hoping that others would not experience it as much as I did. I know other people are also experiencing it, so I reckon everyone who currently is testing is having the same problem.

The core of the problem is that during the init phase of the popup, an error occurs that I can find the origin of. I added that "Something went wrong" message to try and deal with it. I used one of my Apple Code-Level support requests but the engineer thinks it's a bug and told me to file a bug report, which doesn't really help.

In my situation, I chose a different directory than my main userscripts directory on my icloud drive and once that one worked, I subsequently selected my main userscripts directory again, and the error did not always occur. It still sometimes happens, but I am able to refresh the popup (by clicking retry) or by clicking the "refresh" button in the top of popup and it usually goes away.

I am going to try and figure out a way to get around this and push a new build asap.

quoid commented 2 years ago

I just pushed a new build, I expired the previous. This fixes the ui on my iPad and hopefully others, so that should be able to be tested now.

I also include a numeric "error code" that prefixes the something went wrong message, in an attempt to track down what is going on.

Here are some things to try if you encounter the something went wrong message:

refresh the popup by clicking the “click to retry” link
refresh the popup by clicking the “refresh icon” (icon that looks like a circular arrow)
closing the popup and reopening it
closing safari and reopening
trying to open the popup on a different page
selecting a folder with fewer userscripts in it, then switching back to the main userscripts folder

If you are able to successfully run the extension, please let me know. I am not seeing the issue persistently on my iPhone, however my iPhone is "used for development" so I am unsure if that makes the issue less prevalent.

bradenglund commented 2 years ago

I chose a different directory than my main userscripts directory on my icloud drive and once that one worked, I subsequently selected my main userscripts directory again, and the error did not always occur. It still sometimes happens, but I am able to refresh the popup (by clicking retry) or by clicking the "refresh" button in the top of popup and it usually goes away.

I tried this, but am still experiencing the error every time. It only works in a blank directory and then says none found. Otherwise always error.

quoid commented 2 years ago

@bradenglund

I tried this, but am still experiencing the error every time. It only works in a blank directory and then says none found. Otherwise always error.

I pushed a new build, so you should have gotten the option to update. I don't think this will solve the issue, but it might give me more feedback as to which function is failing.

This is an incredibly frustrating error because I can not reliably produce it here, but I am going to borrow another device from a friend to see if I can have a better testing protocol.

bradenglund commented 2 years ago

On new build I a getting error “3 Something went wrong”

On the bright side it appear to function on refresh, on/off and update for a blank directory now.

v7rzlhb3xg commented 2 years ago

Same error, "3 Something went wrong."

Interestingly, after installing it on my iPad, I wasn't getting that error for the first couple minutes. However, it either wasn't able to find any of the scripts in the folder I pointed it to, or it wasn't matching it to the website. Simply put, no scripts were showing up. It wasn't until several minutes later that I got the previous error, though. That being said, the iPad UI works great!

quoid commented 2 years ago

@bradenglund

On the bright side it appear to function on refresh, on/off and update for a blank directory now.

Are you saying it is now working after you refresh?

@v7rzlhb3xg

It wasn't until several minutes later that I got the previous error, though.

When you get the error, does it persist even after refreshing?

3 Something went wrong.

That indicates that the native message call for getting matches failed.

quoid / userscripts

iOS + iPadOS Support #135

145