Why I gave up using iCloud & CoreData (and closed “The Pillow Book”)

It’s sad closing a project in which you have been working for several years, but that’s what I just did some minutes ago with The Pillow Book. I removed the App from the App Store, both for iOS and for OS X. The Pillow Book was one of the first Apps that I designed and built for myself, and one of the first to be published in the App Store under my own name. I loved the App. I was dissatisfied with the diary Apps I found at the App Store at the time, and decided to build my own diary. It would be easy to use, clutter-free and with minimalist design (something that was lacking in every single diary App from the App Store) and would sync the entries among all iOS devices/Mac computers of the user.

This last point probed to be the main problem and eventually the killer of the App. In order to be able to sync among all devices, the only viable alternative I found was a combination of Core Data and iCloud. I had worked with Core Data before, and also with a document-based iCloud App, so I thought that combining both technologies would be easy. Man was I mistaken.

The problem is: iCloud and Core Data just don’t work together. At all. It has been documented, and developers’ discontent against it has been expressed in such notable media as Ars Technica and The Verge. The web is full of stories of nightmarish developers’ experiences trying to reliably use both technologies in their projects.

The Pillow Book was no exception. From the very beginning of the development process, I realized how unreliable and hard to debug was iCloud when working against a CoreData database. The changes on one device would take years to propagate to other devices, and sometimes they won’t even propagate at all, or would break previously committed changes. Error codes were inexistent or totally cryptic, and I had the feeling that I was left alone on my own due to the lack of a serious documentation. I did read lots of books about Core Data and iCloud (almost making me an expert in the process), but most of them would not delve into the hidden details, and would not include anything about issues or problems of real-life usage.

However, because I’m a pig-headed person, I decided to release the App and continue improving it, hoping that I would eventually come with the trick that would make it work flawlessly. And The Pillow Book went live. It had several thousands of downloads, and I even earned some money from it, just to pay some coffees. It had some very positive reviews in the App Store, and everything seemed to go well. But although the App was praised for its design and UI/UX, there were also some negative reviews about nasty things happening to some users. An entry would suddenly disappear for no reason, or it would loose some data, like the entry’s image. These bugs were almost always iCloud+CoreData related, and most of the time I had no idea of what was happening. Then suddenly, after an update, most users started to experience the removal of some or all of their entries, sometimes randomly. This resulted in lots of negative reviews to the App, and deservedly so. If I were a user of a diary App and one day some (or all) of my entries disappeared, I would be very angry.

Captura de pantalla 2014-08-15 a las 23.24.55

Some of these bugs were obviously my fault, and the complexity and unfriendliness of iCloud+CoreData is no excuse for that. However, many bugs were untraceable, and even today, I am not sure why they were happening or how to fix them. The frameworks were really unreliable, and Apple has no excuse for it, specially when even now they are promoting it as a technology that “just works”. Sorry Apple, this time you didn’t get it right.

But my biggest weakness is perfectionism: I am a perfectionist, and I want to be proud of my work. I love working and releasing quality work, so I couldn’t really stand having such a unreliable App. Thus, I have decided to remove it from the iTunes Store and the Mac App Store altogether. I think it’s better this way, as I would not like someone having a less-than-awesome experience after buying one of my Apps.

This also raises an important question for me. If I need to do something similar in the future, I am PRETTY SURE I will not use iCloud + Core Data, but I don’t know of a reliable, seamless alternative. Most folks out there seem to be using a combination of CoreData and DropBox (through 3rd party libraries like TICoreDataSync), but I don’t want to force my users to create a DropBox account (Imagine having to register a DropBox account for just having a diary?). Another alternative seems to be using a Web Service, but I think it’s an overkill having to communicate with a server backend for just having a private, device-shared diary. Furthermore, using iCloud solutions automatically restricts your App to the iOS ecosystem, and that’s not really useful in these Android dominated marketshare days.

Comments(25)

John S
September 21, 2014 At 8:58 pm

I never liked iCloud, it was never that friendly to anything but Apple products. Personally I would choose something outside of the Apple garden. DropBox is stellar at compatibility across platforms, and even Skydrive
and Google’s cloud services are far more open to cross platform compatibility. Apple really needs to open up and realize many users of tech will not simply marry into a Apple ecosystem. They may have multiple devices, OS’s and need more flexibility then a iCloud can provide. Apple continues to limit its success because it cannot let go of its ideal of a everything Apple user.

    Ignacio Nieto Carvajal
    September 21, 2014 At 8:58 pm

    You are absolutely right, John. Apple needs to open some of their technologies, like iMessage, iCloud and such. However, I don’t think Dropbox or Skydrive are good replacements for iCloud. For starters, they need a user account, so from the User Experience perspective, they are not practical solutions, as I don’t want to force my users to open a dropbox account or sign up for skydrive just to be able to use an App that has nothing to do with storage. Besides, most users are non-IT people that have never heard of Dropbox or Skydrive (at least here in Spain). I think we need another kind of solution, one that would allow my mom to use the App without having to resort to 3rd party services.

TimeCurl: Time Tracking made Easy
December 18, 2014 At 8:58 pm

[…] The main problems are sync and conflict issues. You enter new data on device A and do the same on device B, however iCloud is unable to sync it properly – even if there is no data conflict – and you are basically stuck. Another problem is that iCloud would just not sync the data, downstream or upstream, and there are very few possibilities to know why. You will get a basic idea of the kind of problems you can get in the link [1]. While being an older post, it is still very accurate at describing the possible issues. We finally gave up with iCloud & CoreData and came back to a less magical yet functional CoreData local store. Further thoughts about CoreData and iCloud troubles in link [2]. […]

Dmitry
February 16, 2015 At 8:58 pm

I’m switching all my Core Data apps to Realm. Realm has some current limitations, but even with all workarounds it much easier and reliable that Core Data. It has no sync engine, but I believe Realm and Dropbox will work fine together.

    Ignacio Nieto Carvajal
    February 17, 2015 At 8:58 pm

    Thanks Dmitry, I didn’t know about Realm, but I will have a look at it. Anyway, I am more REST focused these days. I think it’s more work and effort, but it’s worth it. Best!

Shawn
March 20, 2015 At 8:58 pm

Have you figured out a better solution? I’m still stuck in iCloud + Core Data with my app (all the similar symptoms), and am trying to play with this Ensembles syncing framework, but haven’t got luck yet. Curious to see if you find better alternatives

    Ignacio Nieto Carvajal
    March 20, 2015 At 8:58 pm

    Hi Shawn,

    Unfortunately not.

    In my case, I decided to abandon iCloud+CoreData completely and use REST interfaces and services for all of my Apps. Not only they are more reliable (and debuggable), but they are multi-platform and useful for Android Apps, Web Apps…

    From my personal experience, I won’t recommend any developer to use iCloud+CoreData.

    Best.

Mark
April 18, 2015 At 8:58 pm

Ignacio,
I just found your post and I couldn’t agree more. This is EXACTLY the same experience I had with my first app that I released a couple of years ago. I ended up just using Core Data on the local device but not syncing to the cloud. I REALLY appreciate your post and would hope someday Apple fixes this for good. It should just work, IMHO, but in practice it doesn’t. Anyway, I am only building for iOS devices so I don’t have to support Android. You say you have moved to REST services. My question is, how have you chosen to deal with the disconnected device problem? I am currently thinking for my next app to use Core Data locally and include the Ensembles framework to do the syncing (I guess I am too stubborn to give up altogether). Yes I know that uses iCloud, but when I last tried this a year ago, it seemed to work better than straight iCloud sync. So when your apps are disconnected from the Internet and therefore cannot talk to your REST service, how have you chosen to persist your local data. Do you see Core Data working locally or have you abandoned it altogether, and if so what did you chose instead? Thanks for any insight.

    Ignacio Nieto Carvajal
    April 20, 2015 At 8:58 pm

    Hi Mark,

    Thanks for your comment. Depends on the App. For your typical social network App, I have decided to show an error message to the user and just ask him/her to try again, hoping that this time there is network connectivity (as they are not “critical” Apps and this behavior is already accepted thanks to Facebook, Twitter, …). On more critical apps where data needs to be saved, I have resorted to local storage in custom data structures, stored in the filesystem as either blobs or json structures referencing blobs when necessary. I have my app check whenever it starts if some data needs to be synced to the REST API. I agree this is not optimal, but makes the App more portable to other systems (android, web app…) and also avoided me of lots of headaches. No more sync nightmares…

    Sorry to hear about your project. I know how painful it is to find yourself in such a deadlock due to the technology chosen. I really wish you the best of lucks in making it work anyway you choose.

      Shawn
      May 4, 2015 At 8:58 pm

      Hi Ignacio & Mark – looks like Mark and I are in the same boat :) Mark – did you have luck with Ensembles? So far I feel it’s worse than iCloud… maybe because I’m not setting up in a correct way?
      Ignacio – did you setup your own server or use some Amazon S3-ish service? Did you have to write a big syncing system, or used some kind of framework? I feel it’d be lots of work to make a syncing mechanism to a server to deal with all kinds of conflict errors, and there should be experts doing it somewhere…
      Thanks!

Ignacio Nieto Carvajal
May 4, 2015 At 8:58 pm

@Shawn: for my Apps I am managing my own VPS (I use DigitalOcean), that are more than enough for casual usage (100-300 users/day) , and I have written kind of a framework (so to say) that makes everything stay synced and under my control (I can choose when and how to sync, and I have control of the storage, success/failure of syncing operations, and the reasons why).
For more serious applications developed for 3rd parties, we usually stick to an Amazon S3 server or similar, but even though I can usually join the technology discussion, the decision is taken at a higher business logic level.

Regarding the syncing, actually it isn’t that hard to make it work once you have built your classes and have control over them. Yes, it is more work, but for me it has been a stress-relief decision, as now I can control and debug the behavior of my apps.
I wouldn’t recommend sticking to 3rd party frameworks built on CoreData/iCloud, because the problem is still there, no matter how good the framework is. Besides, @Mark states that he is not supporting Android devices now, but you never know if you would need it in the future, do you?

    Shawn
    May 5, 2015 At 8:58 pm

    Thanks for the insights Ignacio! Yes making a cross-platform server centric database is an earlier-or-later step to take, just want to find some approach at the moment, but looks like not easy because iCloud’s fundamental flaws :(

Zack
June 11, 2015 At 8:58 pm

I am experiencing same issues with iCloud and Core data. I submitted a crash bug fix release with only one line code change which was 100% not related to CoreData, but I received emails from users saying that they lost all their entries after update. I could not replicate from my own device. It seems like the data still exists on iCloud by looking at iCloud storage size, but it’s just not showing within the App.

    Ignacio Nieto Carvajal
    June 11, 2015 At 8:58 pm

    Sorry to hear that Zack. I have currently switched to different alternatives (REST mostly) and I am really happy I don’t have to rely on iCloud+CoreData anymore. I hope you the best of lucks resolving the situation, I hope your App doesn’t end up like mine.

    Grant
    August 19, 2015 At 8:58 pm

    Hey Zack, I have the exact same problem. Did you ever figure out what was causing this?

Neerav Kothari
November 4, 2015 At 8:58 pm

I have been working on my app that uses a big core data model for over 2 years now. This February I finished integrating iCloud to work with core data. BOY WHAT A NIGHTMARE! everything works fine when icloud is diabled. the moment icloud is enabled, nothing works as expected.

weird errors and exceptions! importing changes is just as u described. data is either lost during sync or simply inconsistent. and i am not even talking about a large persistent store. just a few entries added to test the app.

finally after reading about scores of developers complaining all over the internet about coredata with icloud woes, i decided to drop it.

i am looking for an alternative solution. i want to stick around with coredata and be able to sync my store with another cloud service. my first preference is an authentication less cloud service. but if that doesn’t work i am willing to adopt dropbox.

i am surprised why it doesnt bring any shame to apple to have this service (cd with icloud) around for almost 4 years when it has NEVER worked for most part and for most developers?! apple is acting completely irresponsible about fixing this crap.

i am glad i have still not launched my app as this is my first professional endeavor in life and i didn’t want to be hit hard with negative reviews.

can u please suggest me what would be the best alternative approach if i want to stick around with core data?

i even researched on using icloud document storage to just let the user backup and restore the store ,if not sync it across devices, but even that seems programmer unfriendly. for the life of me i cant figure out how to go about it! and since it automatically is supposed to sync the backed up file across peers, it only means more woes to handle.

please suggest me something as i am many months past my deadline of launching the app.

    Ignacio Nieto Carvajal
    November 10, 2015 At 8:58 pm

    I Neerav, sorry to hear about your situation. I’ve been there before. I won’t really recommend sticking with CoreData, but switching to a more scalable, reliable and cross-platform system such as REST APIs or similar. Maybe there’s a library for translating REST calls to CoreData objects. I myself switched completely out of CoreData, iCloud or similar. I Hope you will be able to find a solution.

    Best regards.

Adrian
January 21, 2016 At 8:58 pm

I’m late for dinner, and you’ve probably already heard about it. But the combination Couchbase Couchbase Server Lite + is the best replacement for Core Data + iCloud. The offline support is fabulous, and synchronization management, even better, fully transparent from the developer perspective. Moreover, it is very easy to install in a dropplet Digital Ocean. Cheers!

Cor Brink
March 19, 2016 At 8:58 pm

Hi Ignacio,
I’ve been running this through my brain for months now and I’m still not sure what to do. Having read your article I would like to ask a bit of advice.
I’m developing an app for iOS and Android devices that stores user input data for statistical purposes in a Realm model on the device. My problem is this; if a user changes handsets all his data is gone and this will not be good for the user or for me…
Questions are:
1. What would you do to store the model data for the user so that he can get it back if he changes handset?
2. Will this solution require to have login details added to the app? (My idea was an internet free app but I guess nowadays it’s not really possible)

    Ignacio Nieto Carvajal
    March 21, 2016 At 8:58 pm

    Hi Cor, if you only need stats about user’s behavior, my advice is to stick to some analytics tool or framework. There’s no need of implementing a full user authentication scheme if you don’t need anything from the users but statistics. Apple and Google both provide libraries and frameworks for that, and there are many 3rd party libraries out there too. Another option is implementing a lightweight REST server to send this interaction to the server. I don’t think that stats are that critical to store locally in a coredata/sqlite local storage for the unlikely event that the user doesn’t have any connectivity.

      Cor Brink
      March 22, 2016 At 8:58 pm

      Hi Ignacio, thank you for the reply. I’ll look into this (have actually) but I’m not keeping record of user behavior.
      What happens is that users input data on a form supplying analytical data (date, place, different types of scores, etc).
      By not having this data stored outside the device (and I guess also for user behavior), this data will not be available when the user changes handsets.
      So the data I’m referring to is not app settings related.

        Ignacio Nieto Carvajal
        March 22, 2016 At 8:58 pm

        Then my advice will be: go for a RESTful API in a backend. It’s the best way to keep data available for all devices in a centralized way. If you add Android to the equation later you will be grateful for having gone in that direction, as the data will also be available for these devices.

        Best of lucks!

Leo kwan
January 8, 2017 At 8:58 pm

Great article, I’m currently facing a similar situation with my app. It was built on Core Data, and I’ve been meaning to add shared device backups. Fortunately we’re in 2017 now and the coredata iCloud api is deprecated, so I know better! Thanks for sharing your experiences!

Ramy
November 20, 2017 At 8:58 pm

I worked with iCloud enough long to understand that it’s just messy and sometimes things don’t work for no reason. But that’s not a reason to give up and remove your app from the app store. You can always rely on other technologies, such as OneDrive or Dropbox.

    Ignacio Nieto Carvajal
    November 22, 2017 At 8:58 pm

    Hi there Ramy, thanks for your comment! Unfortunately, one of the main premises behind “The Pillow Book” was to make it easier to use for non IT savvy users, and that unfortunately disqualifies using any external account or third party service like Dropbox or OneDrive. The app needed to be used by my mother without knowing how to open a Dropbox account.

Leave a Comment

sing in to post your comment or sign-up if you dont have any account.