search

syncthing / syncthing

Open Source Continuous File Synchronization
https://syncthing.net/
Mozilla Public License 2.0
65.87k stars 4.32k forks source link

Native iOS port / GUI #102

Open sloev opened 10 years ago

sloev commented 10 years ago

Developing an ios client for syncthing. right now GO doesnt easily compile using xcode but is compiles manually on-device (requires jailbreak)

What will it take to make an ios client? should it embed the GO syncthing or be a complete rewrite specific for obj-c?

lets discuss!

This issue is closed for discussion. Vote or add bounties etc here: https://www.bountysource.com/issues/7699463-native-ios-port-gui

// @calmh 2016-08-18

calmh commented 10 years ago

That's an awesome initiative!

I don't think it's possible to get an app approved that contains foreign binaries (such as a standalone syncthing written in Go). Go currently doesn't support being linked as a library to other C (or Objective-C) code, due to the runtime. So as it is, I think the only way forward that is guaranteed to succeed on that front is a pure Objective-C implementation.

sloev commented 10 years ago

it would be AWESOME(!!! repeat forever) if you guys included an example of implementing the protocol in another language.

aside yes i guess implementing the protocol in pure obj-c is the way however it seems i have nowhere to start (since lack of third-language implementation).

i am willing to make a big effort on the ios side but the objective would be to keep inside the frames of non-jailbreaked phones and free of course :-)

again "third party" implementation of protocol/functionality is highly needed

calmh commented 10 years ago

Someone has to be the first third party, and it obviously can't be me since I'm the first party. :)

I would start with reading https://github.com/calmh/syncthing/blob/master/protocol/PROTOCOL.md and filing bugs on any ambiguities or other strange stuff.

sloev commented 10 years ago

is there similar docs (have searched but not reaaaaly found any) for the network/transport layers. i mean documentation on the establishment of lan/internet wide connections.

filoozom commented 10 years ago

Are you talking about the discovery? https://github.com/calmh/syncthing/blob/master/discover/PROTOCOL.md

sloev commented 10 years ago

yes thanx. i know its not good githubing to mention issues in issues in .... but that site is not listed from the wiki. :-)

calmh commented 10 years ago

Yeah... Beware that while I always intended the block exchange protocol to be a stable standard, I haven't put nearly as much thought into the discovery protocol. That was always more of a half baked lets-do-something-and-see-what-happens thing that might lead into a DHT or mDNS or something in the long run.

calmh commented 10 years ago

Also a note on the BEP protocol is that the specs kinda vaguely say something about it typically being used with TLS over TCP. I'm considering mandating that or interoperability will be tricky in the future. From the beginning I was considering running it over UDP etc but I'm not sure there is a legitimate use case for anything not TLS/TCP.

sloev commented 10 years ago

yeah i guess an iphone app with "just" the ability to show an index of files using a predefined ip is a starter. however if the discovery is not soon ot be set in stone then i guess it really disturbes development of third party stuff :-)

calmh commented 10 years ago

Agreed. Lets have a discussion about that when the first third party project is at the point where it's relevant. :)

calmh commented 10 years ago

About the discovery specifically, I really feel that is an area where a single app like syncthing should not have to make up it's own thing. There should be a standard ready to use. I think that standard is mDNS w. DNS-SD and that is where I'd like to go. I just haven't had the time / energy / prioritization to go there when a custom thing was easier to get off the gorund.

sloev commented 10 years ago

mDNS / dns-sd is sweet and extremely transparent in that it delivers knowledge on its services and optionally hints on how to use them.

but it only works locally. you still need a standards solution globally.

sloev commented 10 years ago

by reading: https://github.com/calmh/syncthing/wiki/Case-Study i got a better understanding of the global lookup in this case using your announce server. is the announce server sourcecode available?

filoozom commented 10 years ago

It is: https://github.com/calmh/syncthing/blob/master/discover/cmd/discosrv/main.go

filoozom commented 10 years ago

And there are at least two mDNS libraries for Go if you decide to use it for the local discovery: https://github.com/davecheney/mdns/ https://github.com/armon/mdns

calmh commented 10 years ago

Thanks, the latter looks really nice.

calmh commented 10 years ago

It's funny how I went looking after this around new year and then there weren't any good libraries. Now there are. Things moving. :)

filoozom commented 10 years ago

Yes, guess that's always the case on newer languages. For the mDNS thing, of course it doesn't work on Windows... With https://github.com/armon/mdns I get 2014/03/24 17:09:26 [ERR] mdns: Failed to start IPv6 listener: listen udp6 ff02::fb: setsockopt: not supported by windows

EDIT: Never mind, works over IPv4, and it somehow still sends IPv6 paquets despite the error.

calmh commented 10 years ago

That's a bit annoying. I wonder how much is Go just not supporting Windows and how much is just not using the right calls within Go...

calmh commented 10 years ago

It should be fixed in either case, just different places to file the bugs I guess.

sloev commented 10 years ago

Yes mdns does need external stuff. Bonjour tools i think. But that had been talked thru 1000s og times. On Mar 24, 2014 5:11 PM, "Philippe Schommers" notifications@github.com wrote:

Yes, guess that's always the case on newer languages. For the mDNS thing, of course it doesn't work on Windows... (at least not without an external program if I understood it right).

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38463953 .

filoozom commented 10 years ago

For the moment syncthing is standalone, so I don't know if it's good to add a dependency?

sloev commented 10 years ago

Dont worry people use banjo ur for Windows all the time for different services like printer discovery,airplay etc. On Mar 24, 2014 5:31 PM, "Philippe Schommers" notifications@github.com wrote:

For the moment syncthing is standalone, so I don't know if it's good to add a dependency?

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38466560 .

filoozom commented 10 years ago 
var ipv6Addr = &net.UDPAddr{
    IP:   net.ParseIP("ff02::fb"),
    Port: 5353,
}

_, err = net.ListenMulticastUDP("udp6", nil, ipv6Addr)
if err != nil {
    fmt.Println(err)
}

returns listen udp6 ff02::fb: setsockopt: not supported by windows, so it's neither the library nor Go, it's just Windows?

I personally don't have Bonjour installed and still have a lot of programs on my computer. The only program I'm aware of that uses Bonjour is iTunes and apple programs, which I don't use.

calmh commented 10 years ago

Yeah, I wouldn't want to force that (Bonjour and co) on anyone either. Do you have v6 enabled (or not disabled, or whatever :) on that computer? If so, that means the syncthing 0.6.6 discovery doesn't work on Windows either, again. Sorry about that. :(( Will need to either figure out what the problem is in Go or not use v6 or find some workaround. This is all not so related to iOS anymore though. :)

filoozom commented 10 years ago

Yes, I have IPv6 enabled, but I can test on an other computer.

net.ListenPacket("udp6", "[ff02::]:21025")

works

net.ListenMulticastUDP("udp6", nil, "ff02::fb")

doesn't work

Globally I think net.ListenPacket on IPv6 is not a problem, but net.ListenMulticastUDP is. So maybe there's no issue in the discovery.

sloev commented 10 years ago

Implementeret both and Add a token to the config whether a node Accepts mdns. Mdns is great. Keep your own discover going for legacy (Windows) support. On Mar 24, 2014 5:48 PM, "Philippe Schommers" notifications@github.com wrote:

Yes, I have IPv6 enabled, but I can test on an other computer.

net.ListenPacket("udp6", "[ff02::]:21025")

works

net.ListenMulticastUDP("udp6", Iface, ipv6Addr)

doesn't work

Globally I think net.ListenPacket on IPv6 is not a problem, but net.ListenMulticastUDP is. So maybe there's no issue in the discovery.

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38468934 .

filoozom commented 10 years ago

But why should there be two discovery systems when the current one works for all operating systems (I mean maybe, still not sure that it works correctly on Windows)?

sloev commented 10 years ago

The only reason is that third party apps using your protocol on linux/osx has one less custom protocol to navigate. Mdns is heavily used in linux/osx. On Mar 24, 2014 6:05 PM, "Philippe Schommers" notifications@github.com wrote:

But why should there be two discovery systems when the current one works for all operating systems?

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38471157 .

filoozom commented 10 years ago

Yes I know, but if third party apps want to interact with syncthing running on Windows it wouldn't work, which doesn't solve the problem, neither for Windows nor for the third party app developpers.

sloev commented 10 years ago

I guess i have to announce my strong advocation for development of a syncthing lib for integration into "your own" apps. Is that lining up good with you guys? On Mar 24, 2014 6:09 PM, "johannes Gaardsted Jørgensen" < johannesgj@gmail.com> wrote:

The only reason is that third party apps using your protocol on linux/osx has one less custom protocol to navigate. Mdns is heavily used in linux/osx. On Mar 24, 2014 6:05 PM, "Philippe Schommers" notifications@github.com wrote:

But why should there be two discovery systems when the current one works for all operating systems?

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38471157 .

sloev commented 10 years ago

This lib could og course also include your discover protocol. However it adds to the complexity og grasping the lib plus it recreates functionality allready existing for devs using syncthing between unixes only. I am not against yor discover thing i am just having a hard time allready trying to understand just the protocol part :-)

Btw good project and nice work! :-) On Mar 24, 2014 6:11 PM, "johannes Gaardsted Jørgensen" < johannesgj@gmail.com> wrote:

I guess i have to announce my strong advocation for development of a syncthing lib for integration into "your own" apps. Is that lining up good with you guys? On Mar 24, 2014 6:09 PM, "johannes Gaardsted Jørgensen" < johannesgj@gmail.com> wrote:

The only reason is that third party apps using your protocol on linux/osx has one less custom protocol to navigate. Mdns is heavily used in linux/osx. On Mar 24, 2014 6:05 PM, "Philippe Schommers" notifications@github.com wrote:

But why should there be two discovery systems when the current one works for all operating systems?

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38471157 .

calmh commented 10 years ago

@filoozom I read you loud and clear - custom and working beats standardized and not working, every time. :)

@sloev That would be nice. Such a library is out of scope for syncthing though, that would be a separate project. One that I hope happens, one day.

sloev commented 10 years ago

To take it back to ios... The lib IS my scope since go and ios is not in a good relationship. I have to write the discover thing and the protocol from scratch refering to your dox and code. And this is my angle since the ios supports mdns that would have been easy to resolve IPS but instead i have to write the custom discover. But this can be Solved colaborately by asking you guys questions on discover and protocol. :-) So question 1: when TCP is established is TLS established with starttls? On Mar 24, 2014 6:45 PM, "Jakob Borg" notifications@github.com wrote:

@filoozom https://github.com/filoozom I read you loud and clear - custom and working beats standardized and not working, every time. :)

@sloev https://github.com/sloev That would be nice. Such a library is out of scope for syncthing though, that would be a separate project. One that I hope happens, one day.

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-38476545 .

filoozom commented 10 years ago

Are you talking about the discovery or the actual block transfer? There is no TLS in the discovery.

sloev commented 10 years ago

i was talking about this: "Compression is started directly after a successfull TLS handshake, before the first message is sent. The compression is flushed at each message boundary." do i need the generated cert from client b in order to do handshake with client a?

On Mar 24, 2014, at 20:09 000, Philippe Schommers notifications@github.com wrote:

Are you talking about the discovery or the actual block transfer? There is no TLS in the discovery.

— Reply to this email directly or view it on GitHub.

filoozom commented 10 years ago

It's a direct TLS 1.2 connection and then you just need to make a certificate request.

calmh commented 10 years ago

Yep. Each side needs it's own key and certificate. These are regular RSA certificates. The connection establishment goes like this;

If you want examples of the certificate and key the easiest way is to let syncthing generate them. You'll already have a pair in ~/.syncthing. Move them somewhere else and a new pair will be generated on the next startup. It also prints the node ID so you can verify that you get the same results if you like.

calmh commented 10 years ago

I'm going close this issue (since it's not something to be solved in this repository) and point to the forum at http://discourse.syncthing.net/ where I'll be happy to continue the discussion. And I mean that, I love the discussion, just not in the issue tracker. :)

sloev commented 10 years ago

Yeah do that and sorry it went on here instead og the right place :-) On Mar 31, 2014 11:49 AM, "Jakob Borg" notifications@github.com wrote:

I'm going close this issue (since it's not something to be solved in this repository) and point to the forum at http://discourse.syncthing.net/where I'll be happy to continue the discussion. And I mean that, I love the discussion, just not in the issue tracker. :)

Reply to this email directly or view it on GitHubhttps://github.com/calmh/syncthing/issues/102#issuecomment-39070966 .

marclaporte commented 9 years ago

For the record:

Syncthing is now MPLv2 licensed https://discourse.syncthing.net/t/syncthing-is-now-mplv2-licensed/2133

One of the goals of this change is to facilitate having Syncthing in app stores.

mailinglists35 commented 8 years ago

this seems a real killer app but no IOS client makes it unuseful to me

calmh commented 8 years ago

Reopening for visibility and to enable voting / bountying. I still consider it unlikely that this will happen as part of Syncthing "core", but having an issue for it somewhere seems appropriate.

AudriusButkevicius commented 8 years ago

I think the effort of making an iOS port would be better replaced with a Xamarin port which targets more platforms.

calmh commented 8 years ago

https://www.bountysource.com/issues/7699463-native-ios-port-gui

signmeuptwice commented 6 years ago

I would love to work on that. Is there any doc on the server api to develop the client ?

Swift is the way to go there.

AudriusButkevicius commented 6 years ago

Docs site has the docs...

signmeuptwice commented 6 years ago

yeah saw that right after asking... stupid question :)

hyiltiz commented 6 years ago

It would be extremely sad to re-implement the whole library in Ocjective-C just because its language was not allowed/restricted by some company. Going the other way, isn't there any way to petition it s.t. Go will be somehow supported in the iOS "ecosystem"?

AudriusButkevicius commented 6 years ago

It's not really restricted, you can compile go to C compatible ABI, and even access parts of ObjC/iOS SDKs from go via go-mobile, etc.