![]() Go excels in concurrency which is really important for rclone too. We’d been using Python a lot at work so the single binary for a deployment was just breath of fresh air. There were some things that attracted me immediately to Go. I need a program to exercise that “swiftsync” which became “rclone”. I made a few test programs then I wrote my first major go library, an interface for the openstack swift object storage system. In 2012 I decided to learn Go as it had just reached the exciting milestone of a 1.0 release. All except Windows and macOS which I had to build natively to fix various problems.Īs I began to do less programming at work, I realised I needed to do some programming at home in order to retain my sanity. Nearly all the platforms are cross compiled on a linux host as part of the Travis build. Rclone is licenced under a very liberal MIT licence which means that it gets bundled into Synology NAS and other products like that. Thanks to Go’s amazing cross compiling abilities, rclone runs on nearly every platform you can think of. I particularly like the HTTP backend – you point it file listing webpage (for example the ones apache generates) and it can pull the files off there recursively, or you can mount it as a disk.Īll rclone operations are supported by all providers. There are also the standard protocols like FTP, SFTP, HTTP. Well I think so, they keep making new ones!Ĭertainly all the big names are here, both for business use like s3, google cloud storage, azure blob storage and for personal like google drive and dropbox. ![]() Rclone supports most major cloud providers. It would be possible to implement it but you’d lose that 1:1 mapping of objects. That unfortunately isn’t possible when trying to copy objects 1:1 to cloud providers as they just don’t provide the API to do it. There is one thing rclone does not do though and that is implement the extremely clever delta encoding that rsync uses for transferring objects. I’ve tried to keep the flags the same where possible to make it familiar to rsync users. Rclone syncing works very much like rsync. It has some optional features like encryption, but by and large rclone maps objects 1:1 from source to destination. It can transfer data directly between cloud providers so you can use it to back up your s3 objects to google cloud storage. ![]() Rclone takes lots of care with data integrity, checking hashes at every step and preserves timestamps on your files. In summary rclone is a command line program to sync your data to and from cloud providers. Oh and thank you for everyone who has sent a PR correcting my spelling :-) What is rclone? I’d like to thank my excellent core developers, the super helpful group of people on the forum and anyone who has sent a PR or raised an issue – rclone wouldn’t be what it is today without you. I’ll then discuss how you test a project like rclone and round off with a look at some of my favourite libraries.īut before I get going I don’t want you to get the impression rclone is all my work. I’ll dive into how it works, and show a bit of code. My talk this evening is going to start with a quick summary of rclone and its history. I used to be a data hoarder, but the house filled up with servers and disks and my wife complained about the electricity bill so I’m mostly reformed now. I’ve always had a keen interest in storing data, keeping it safe and transferring it about – hence rclone which is an embodiment of those things! I don’t get the chance to do a lot of coding at work any more so I now do Open Source coding as a hobby. Now-a-days I’m CTO of Memset Ltd who do cloud hosting and storage. I’ve loved programming ever since I was 13 with my ZX80 and have been doing it ever since. Rclone is an Open Source command line program to transfer files to and from cloud storage started by me as a hobby in 2012.įirst a bit about me. I’m Nick Craig-Wood and I’m here to talk about rclone. It's about rclone and how rclone uses the Go programming language. You can watch it on YouTube (26 minutes) or see the slides and read the words here. This is a transcript of a talk I gave at the Go London User Group on.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |