YALA: Yet Another Link Aggregator

Not sure what to do first?

1. Create your account (or login).
2. Click on the Create New Community button (or this link)
3. Click on the Create New Post button, and add a post.

That's it! You just created a community and posted on YALA!

Ready for Users!

Howdy. YALA is in super-early alpha stage. You'll find items marked as "FIXME" or hit with a strikethrough. These are placeholders to indicate that the functionality is not yet developed, but it will be soon.

In the meantime, feel free to go around and kick the tires! Try:

• /c/yaladev - Yala Development Community
• /allposts - a list of all the posts (so far)
• /allcommunities - a list of all the communities (so far)

Check out the Github!

This is the github link for the sourcecode - if you find an issue, feel free to report it!

Want to help out? Check out our list of issues on Github and submit a PR!

Who are you?

I'm Kerry Boyko. I'm a senior software engineer with eight years experience, but I had to move back from the UK to the US to get some medical care. I also had to leave my job to do so, so right now I'm spending much of my time looking for a new one. This is my side project, a built-from-the-ground-up clone of Reddit. (Which itself could have been said to be a built-from-the-ground-up clone of Digg, if I recall correctly...)

The idea is not to create a toy project that can show you can "code" (if I was fresh out of school, I'd have a number of such projects), but this site has been built to scale. This is because I wanted a challenge like those I am currently facing in my career. It's designed to keep my skills sharp and learning new technologies and new approaches to application development.

Why another link aggregator?

Honestly, because I think we kind of need one.

I really adored Reddit. It was an amazing way to learn about all sorts of projects, to connect with community and like-minded individuals. Yes, there were some cesspools on there, but some bright spots too.

About a month ago as of this writing (July 2023), I deleted my Reddit account of 15 years, in response to how the company behind Reddit decided to respond to people who had concerns about the API changes. Personally, I couldn't care less about the API changes, but the idea of forcibly removing moderators from the communities they built from the ground up, was insane, especially considering that the only value that Reddit had was created not by the company's technology but by the moderators and users of the site. (Nobody goes to Reddit to ooh and awe at the source code, or ponder the database, not even me!)

Indeed, this was a known problem in social media which was documented in an article in Wired Magazine by Cory Doctorow, entitled (forgive the language), "The 'Enshittification' of Tik Tok". From that article:

Here is how platforms die: First, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the value for themselves. Then, they die.

All of that seemed to be what happened to Digg - and one of the reasons that Reddit overtook Digg. It also seemed to happen with Facebook, with Amazon, and it certainly has happened with Twitter.

I thought I could do better.

Right, but why another link aggregator?

Okay, I get that. There are some absolutely well designed applications - Lemmy and Kbin among them, that, like Mastodon for Twitter, use the Activity Pub standard in order to create a federation of interrelated link aggregators.

And I like them, but I found them frustrating to use, even though I'm kind of a power user. Say what you want about it, but centralization was the killer app of Reddit. Discussion forums have been on the Internet since, well, before there was an Internet, in the form of dial up BBS boards. But you had to know the number to dial in, and you were restricted by location (or maybe not if you were really good with that Captain Crunch whistle and could make free phone calls.)

Those BBS boards gave way to Usenet newsgroups - a centralized way to communicate. And there was a lovely time as people had fun with alt.swedish-chef.bork.bork.bork. However, sadly, Usenet's technology didn't withstand the spammers, becuase it lacked tools for effective moderation.

From Usenet newsgroups came BBForum and other web-based forums which could be effectively moderated, but once again were hard to find, spread out, and you didn't have a central identity. You also had to know exactly where you were going.

Reddit's big draw was centralization with moderation. Like Usenet, everything was in one place. Unlike Usenet, things could be more or less moderated. Like the bulletin boards and BBS boards, they could cover every topic under the sun, but you only needed one login to start posting about every topic. Indeed, chances are if the topic existed, there was a forum dedicated to it.

What Lemmy and Kbin are trying to do is not centralization but federation, and while federation is, in my professional opinion, really neat, it isn't centralization. You can kinda-sorta have one login, but because you have one login, that doesn't mean you can easily interact with people on other Lemmy or Kbin instances as easily and seamlessly as interacting on Reddit. It can be hard to have the same experience as everyone else, as different servers have different rules about what data they pull down from other clients. In other words, it's just... not the same.

Not to mention that it suffers from what my friend Andy calls the "One Guy In Kansas" problem. It costs money to host a federated instance, and you are at the whim of the person hosting the service to keep your account and connections. If they're doing it for fun, the site will go down when it stops being fun and starts being work. If they're doing it for money, the site will go down when the money stops flowing.

Federation is a way to create a robust network out of individually fragile instances; but nobody wants their user experience to be "individually fragile."

The problem with Reddit was not technological. The problem was the business model. I think I might have a way to solve that.

Okay, what do you mean, "business model?"

All of these social network sites tend to engage in Doctorow's 'enshittification' cycle because they are operating in order to maximize profit. And maximizing profit means that (in the words of Cat Valente), the push once the social network is established as a forum of conversation is to tell users to stop communicating and start spending money. That's really what the API drive of Reddit was about: you provided value to us for free, now we're going to charge you to provide value to us.

From Cat Valente:

Stop talking to each other and start buying things. Stop providing content for free and start paying us for the privilege. Stop shining sunlight on horrors and start advocating for more of them. Stop making communities and start weaponizing misinformation to benefit your betters.

It’s the same. It’s always been the same. Stop benefitting from the internet, it’s not for you to enjoy, it’s for us to use to extract money from you. Stop finding beauty and connection in the world, loneliness is more profitable and easier to control.

Stop being human. A mindless bot who makes regular purchases is all that’s really needed.

It's hyperbole, but not by much.

The interesting thing about all of this is that there was one, very well known social media site, where people can and do collaborate, where they do provide value, where they, despite the fact that human beings tend to screw up things that they touch, has actually created something good and decent and grand.

Wikipedia.

And that's because Wikipedia is run as a non-profit company. They don't have to answer to shareholders as to why they haven't maximized profit, so they can make concious choices not to run ads, to turn down money from interested parties who wish to puff themselves up or detract from their competitors. Somehow, the Wikimedia foundation has made things actually work.

Once this site has reached what I would consider to be a "minimum viable product", I intend to launch it and then set it up as a non-profit using the Wikimedia Foundation model. I have no idea if it will succeed.

Right, so, what does that mean?

Right now - nothing much. Sign up, try it out. Kick the tires. Hopefully this place will eventually get some critical mass and we'll see what happens from there.

Tech stuff

Can I fork the source code?

By all means! Here you go!

Will you be adding ActivityPub?

Not for the minimum viable product. There may be a way to integrate existing ActivityPub published entities with Yala, but if so, the idea would still be to make it operate seamlessly with the rest of the site, so that we don't lose the benefit of centralization.

What's your tech stack?

• This is being built on the Remix Supa-Fly stack.
• Remix is a front-end/back-end framework that uses the technology behind React Router to create a server-side rendering framework for React that supports nested routes. It is similar to, but distinct from, Next.js.
• The main BaaS (backend as a service) provider is Supabase, chosen because they have simple authentication using JWT tokens and provide a Postgres database.
  • PostgreSQL Postgres was chosen both because of the relational nature of conversational data and the ability to handle JSON data, which means that altering future schemas for future needs won't be as challenging compared to something like MariaDB.
  • The auth services are also provided by Supabase and rely on cookie storage of JWT tokens, which are passed in the header of the user's requests (when logged in).
• The ORM used for that database is Prisma, (which honestly, coming from Knex.js and raw SQL, feels like cheating!).
• Deployment is on Fly.io.
• The base UI framework is using Shadcn/UI, a command line tool based on Radix.
• Styling is done using SCSS. Tailwind is installed to support Shadcn/ui, but I don't like it and I don't use it in my code.
  • Thanks to Yomesh Gupta for his article on devtools.tech: "Setting up SASS with Remix Run"
• Unit Testing will be done via Jest; integration testing via Cypress.

Special Thanks:

Special thanks go to the Remix Discord Server for helping me with some thorny issues getting used to the framework.

Thanks also go to my mom, who doesn't understand why anyone would want to use a site like this, but helpfully caught two misspelled words in this About page.

And thanks to Andy, who introduced me to a very similar project with a very similar model.

Current Roadmap

Last update: August 24, 2023

FEATURE FREEZE

• Release 0.1.0 ALPHA;

Quite frankly, I've put off doing this for long enough. We're now in a spot with the site where it can be considered to be more or less feature complete, even if it's not 100% where it needs to be with regards to login sessions, e-mail services, or etc.

Right now, this site is less of a viable product, and more of a tech demo while I'm applying to jobs. The problem is - this tech demo is not 100% where I would like to show the code. First, there's a lack of unit testing and integration testing. Those were put to the side to deliver the features, but it can be put off no longer.

Secondly, there are a number of scenarios where I'd like to simplify and organize the code base. A lot of reusable database calls are in the /routes folder, when that really should be a 'seperate concern'.

And finally, the SCSS files could use simplification and reorganization.

So for that reason, I'm calling a feature freeze right now and calling what I have Release 0.1.0 ALPHA and get the site ready for Release 0.2.0 BETA.

Bugs Found

• 🔲 If the user is writing a reply and the token expires behind the scenes, the application, instead of trying to renew the token, will spit them out to the /login page to get that token refreshed. This means the user loses data they have entered. Annoying.

Bugs Quashed

• ✅ Anonymous Users (who cannot vote) have shown that they have voted - visual bug. -> Anonymous users now cannot vote and are not shown otherwise. Voting has a loading spinner to indicate processing time.
• ✅ Markdown does not render properly in Post Summary View. -> Markdown now renders properly in post-summary view.
• ✅ When a token expires, it doesn't notify the user. It should automatically refresh instead of kicking the user back to the /login route.
  • This bug was caused because "remix-superfly", set the auto refresh and persist values manually to 'false', overriding the default of 'true'. (Cue the Simpsons meme: "Here's your problem, someone set this thing to evil.")
• ✅ Elements now have loading spinners to indicate status (and prevent confusion as to whether the user needs to reload the page or click again.)
  • ✅ Voting now has a loading spinner
  • ✅ Creating communities now has a loading spinner
  • ✅ Creating posts now has a loading spinner
  • ✅ Creating comments now has a loading spinner
  • ✅ Creating replies now has a loading spinner
  • ✅ Subscribing now has a loading spinner

Completed:

• ✅ Signup and Login via Email/Password (basic, happy path)
• ✅ Signed in users can create communities.
• ✅ Signed in users can create posts on communities
• ✅ Signed in users can create comments on posts
• ✅ Signed in users can create comments on other comments
• ✅ Authors (but not moderators) can now edit posts.
• ✅ Authors and moderators can now delete posts.
• ✅ Login via username and password
• ✅ Login via Google Auth (and should be simple to do other Oauth providers)
• ✅ When a user logs in for the first time with GoogleAuth, they are prompted to choose a username so that they can contribute on the site.
• ✅ Created separate Supabase projects for production and development - a "Kick the tires" methodology. The sooner I can get real people to look at the bugs on the site, the sooner I can continue working on them, so I'm creating two DBs, one for production and one for development.
• ✅ Reconnect previously written functionality using new Auth provider. (Long story short: this was originally written using FusionAuth as the auth provider but Supabase was just a better solution. Rather than try to add Supabase to the existing repo, it was just quicker to create a new Remix Supa-Fly project.)
• ✅ A moderation tool to ban/unban users from communities
• ✅ Users can vote on posts, and posts display how the user voted, as well as the total of all votes
• ✅ Users can vote on comments, and comments display how the user voted, as well as the total of all votes
• ✅ Banned users cannot create new posts, cannot create new comments, in communities where they are banned
• ✅ Users can reset their password via 'forgot-password'
• ✅ Authors can edit or delete comments
• ✅ Moderators can delete, but not edit comments.
• ✅ If a comment with children is deleted, the comment is marked as deleted, but not removed in the comment hierarchy.
• ✅ Users can now add images to their posts via URLs. (but not yet edit them.)
• ✅ Authors can now edit post images as well as text.
• ✅ Visitors can now see a list of most popular communities by subscribers, and posts.
• ✅ Visitors can now see a list of most posts (by 'hotness', most recent, etc.)
• ✅ Users creating a community can upload a jpg/png/webp/gif header image to be stored on our servers.
• ✅ Moderators can now edit/update the description of their community, as well as the header image.

What's immediately next;

• 🔲 As a developer, I'd like Unit & Integration testing. (Normally I do TDD first, but I needed to get used to the Remix environment and see what features are feasible.)

What's next?

• 🔲 As a site owner, I would like to require email confirmation for signups before a user can post.

• 🔲 As a user, I would like to place "Reactions" (like Slashdot moderation) for posts and comments

• 🔲 As a user, I would like to be able to tag posts and comments with arbitrary hashes.

• 🔲 As a user making a post (but not a comment), I would like to be able to embed youtube videos.

• 🔲 As a user, I'd like to be able to get a share menu for posts (a popover) which allows me to copy a link to the clipboard or to have a downloadable HTML code that I can embed in a webpage.

• 🔲 As a user, I would like to have a way to semi-privately message other users, perhaps using real-time chat.

• 🔲 As a user, I would like to have a loading notification when I switch routes and load a new website. This should be available through Remix's useNavigation() hook.

• 🔲 As a moderator, I would like to 'lock' posts so that no more edits or comments can be made.

• 🔲 As a moderator, I would like to 'sticky' posts so that they always show up at the top of the page.

• 🔲 As a moderator, I would like to add other users as moderators

• 🔲 As a developer, I would like to have a blog system that uses markdown so that I can have a development blog.

What is being postponed?

• 🔲 Error boundaries need to be configured and prettified.
• 🔲 I may offer support for private communities as a subscription service.

Refactor Targets

• 🔲 SCSS files are servicable but might be simplified for reuse. Undecided if we should just clean up the Scss itself or if Styled Components can be used with Remix's idiosyncratic CSS bundling while still retaining the performance benefits.