Open Thoughts

Musings about open, semantic web tech by Matt Derocher

There is no lack of Tools for Thought (TfT) applications today, but all of them require you to store all your data in one application. What if we could figure out a way to use different apps for different tasks but work from the same documents and data?

Motivation

I am a history buff. Not in the normal way of going to museums or watching documentaries, but in the way of searching for hours through digital newspaper archives looking for certain information to answer a question and then writing about what I found.

I used to use Evernote as a system for managing research for history writing. Evernote has an excellent web clipper, so it was really easy to get things into it. It was not so easy to do anything with the notes and files when they were inside it. I tried creating an elaborate system of notebooks (folders) and tags and internal links, but it became obvious that the app was not meant for that level of use.

Evernote can store any kind of file, but it can only open a handful of file types. I would open the different file types in separate reader apps, each of which had a unique annotation system. I was able to get Evernote to pull in most of the annotations using various hacks, but the sync was one-directional and the annotations were not linked back to the source.

I built a system on top of Sanity, which is a hosted, headless CMS. It was really easy to create data types, so I could have entities like people, places, or dates that could be linked to in blog posts. They have a powerful query language, so I was able to list all posts that mentioned an entitiy. I created an auto-fill component in the CMS that would let you search Wikidata for an entity and it would populate information, like name, date of birth, and a description. I also started building a web clipper that would help me collect data into the system.

At first, I liked the system and was able to use it post some research, but I eventually decided to move away from this solution, too. Besides some quirks and limitations in the API (some of which have been fixed since I tried the project), the main issues I had with the platform were that it wouldn't work offline and the only option was for Sanity to host your data.

Guiding ideas

My goal is for researchers to be able to collect digital sources (websites, documents, media files, and named entities), make annotations on any file type, use these sources and annotations in their writing, and then link back to the original sources.

Avoid lock-in

I have heard about some powerful and popular Tools for Thought applications. I haven't deeply used any of them, because, after my experience with Evernote, I am afraid of getting locked into something that doesn't quite meet my needs. I want to explore ways for people to be able to use my apps without feeling that it is all or nothing.

Small apps for specific tasks

In the old days when we had all our files on our local hard drives, we could use many small applications that had very specific uses. I want to try to capture that flexibility and interoperability between apps. Instead of creating one app that does all the things I need, I can create smaller apps that handle specific activities but can all understand the same files and data.

Smaller apps also mean faster development time and they avoid the complexity and slowness that can come with apps that try to do too much.

Interoperability

If each tool is opt-in, then other developers could create apps that work in the ecosystem. The user could use them without having to stop using the other apps they have been using. To increase this possibility of interoperability, as much as possible, I want to take advantage of standards like RDF or JSON-LD.

Components of the ecosystem

These are the current pieces of the ecosystem that I am working towards:

  1. Web Clipper – Save web pages, PDFs, EPUBs, and media files. I have started on this and have a basic implementation (almost) working.

  2. Reader – Read all different files with user preferences and the ability to highlight and annotate. Annotations also can include Named Entity Recognition to find names of people and places in the content. I currently have a basic reader (without annotations) combined with the clipper, though they will probably be separated at some time.

  3. Annotation manager – A way to view all your annotations in one place. This would probably be a stepping stone to the studio, as mentioned in the next point.

  4. Authoring Studio – Write and combine annotations into notes or blog posts. I imagine something close in concept to current TfTs like Roam or Tana.

Besides these, the possibilities are many, including Dropbox-like local sync or an app that would take a video or audio file, create a transcript with voice-to-text, and then allow annotations on the content.

Technical foundations

  • Content addressing (IPFS, WNFS) – If I have a copy of a PDF and you have a copy, then we can “talk” about the same file. This will also increase the lifetime of a source document because multiple people can host the file (this cuts down on link rot).
  • Stand-off properties (atJSON, W3C web annotations) – To preserve content addressing, we want to manipulate source documents as little as possible. Stand-off properties allow each user to have their own annotations on a shared document. These properties are essentially optional layers on top of the source document.
  • Semantic data (RDF, JSON-LD) – There have to be ways for other applications to know what is being talked about. It will also allow you to find relations between items that you have collected.
  • Collaboration/interoperability (Activity Pub, Linked Data Notifications) – This would allow people to join groups and get notifications when someone made annotations on a document I have an interest in.
  • Offline access (WNFS) – It is important for me that apps are fast and accessible at all times because I spend a lot of time in Africa where there is metered and sometimes spotty network access.

Conclusion

I have been working on this part-time as a side project when I am not working for my day job as a front-end developer. Inspired by Linus Lee's ideas of creating tools that you would use yourself, I am trying to get to what I am calling MMP (Minimum Matt Product) so that I can use these tools in my research and everyday reading. If I can get it to where I will use it every day, then I will keep working out the details so that it can be usable by more people.

The promise of technology

There was a time in my late teens/early twenties when I was enamored by technology. I had swapped out my Windows laptop for a Macbook Pro. Every week, I eagerly awaited the new episode of the Mac Power Users podcast. While waiting for the new episode, I would work through the backlog.

I was fascinated by what were called “workflows” and how different apps could be automated and pass data back and forth between each other. This was achieved using applications like Automator, TextExpander, Hazel, and Alfred. These apps came with a semi-hefty price tag (at least it seemed so for a teen that worked at Subway), but it was a one-time price, so if you bought it, you owned it forever. Sometimes upgrades to major versions cost money, but you could opt out and still use the old version.

Spotlight indexed everything on the computer, so I could search and see my Evernote notes right next to my emails and Pages documents. Apps like Picasa and iTunes also indexed your photos and music, respectively. You had all your data in your control, and you could use a Time Capsule and/or Carbon Copy Cloner to seamlessly make sure if something happened to your computer, you wouldn’t lose your files. (I used both and also Carbonite for off-site backup.)

Dropbox came out, and it was like magic! You could sync all your files between devices. You still maintained full control over them. Mobile apps would connect to the Dropbox API, so it would even (sorta) work on your phone and tablet.

Broken promises

Fast forward to today, and I’ve lost almost all of that fascination with technology. I work on the web, so I am at a computer more than ever before in my life. I switched my laptop back to Windows partly because I refused to pay the “Apple tax” and partly as a protest against Apple’s ethos of a walled garden.

I don’t buy a new phone every year anymore, either. I’ve had the same phone for three years, and I’m hoping to get at least one more year out of it.

There are quite a few reasons why my feelings about technology have changed. Some of them are personal, and some are because of how technology evolved.

Personal growth

I’ll start with some positive reasons. It’s kind of simple: I’ve matured as a human being. I am more thoughtful about how I spend my money. My three-year-old phone does basically all that a new phone does. It has a good camera, and people still give me compliments about how good the camera is. Unless it breaks or you have something significantly different to offer, why would I upgrade? (I have been tempted more than once to buy a foldable phone because it is extremely different than what I have.)

I also want to spend less time on technology. I don’t want to be identified by technology. I don’t want to be either “the guy who’s always on his phone” or “the computer guy.” (Not really positive that I’ve succeeded in this, but I’m trying in some ways.)

The Internet changed everything (and not all for the good)

Going back to talk about the tech itself, everything is different because the Internet has gotten so much more powerful. You don’t need to have your laptop do a bunch of automation stuff anymore because everything is done on the web. In fact, as things move to the web, there are some things that are no longer possible to manipulate on a computer. Companies like Twitter and Facebook have changed how their APIs work, so what can be retrieved from them is limited. We have online-only file formats such as Figma that can’t be accessed without the Internet.

Instead of controlling your data and syncing it with a service of your choice, companies all have their own proprietary ways of managing your data.

Because they have control of your data and you can’t easily access it without the company, they can charge money each month for you to be able to access your data. (Is it an exaggeration to call it ransomware?) Every service charges for their one little utility, and at even $2.99 a month for a service, in less than a year, it is much more expensive than most of the one-off apps you bought for the laptop. Surprisingly, even though they already charge a subscription price, many also sell your data to advertisers.

There is web automation in the form of IFTT and Zapier, but they don’t really match the power of Hazel and the like.

Not to mention, web stuff seems really buggy. It is never finished, so it seems to be in a constant flux of features being added that also add bugs.

Making new promises

All that said, I haven’t given up on technology. I may not like where it is, but I am hopeful for what it can become. The web is powerful, and it opens up possibilities that weren’t available when things stayed locally on our separate computers. It’s going to take time for things to be as seamless as they felt on local computers because things are a hundred times more complicated on the web. There are issues of security, scale, and interoperability. And, yes, these things will take time and effort to figure out, so there will need to be a constant stream of money to make it progress. (Exactly how and to whom this money should be directed is a large discussion in itself.)

I still listen to technology podcasts (and read blogs and books), but the focus isn’t on using existing tech and apps. But it is on how we can make better tech that solves these problems. There are many smart people looking to solve these problems. They want to help the web reach its potential.

I am starting this blog because I want to bring more notice to the people working on these things and also add my ideas to the conversation.

Enter your email to subscribe to updates.