Now with more and more #instances upgraded to version 4.2.0 of #Mastodon, you can compare full text #search on different ones: Having multiple accounts, I did – and yes, the differences are rather big.
Once again it seems clear, the bigger your instance the better for finding content in the #fediverse. Especially for journalists, this also implies, that an instance by your organization might not be the best idea. Something like journa.host could make more sense.
1) You can make even a single user instance have a wider reach through relays, especially targetted ones that are on the topics you're interested in. (More info at https://fedi.tips/using-relays-to-quickly-expand-a-servers-view-of-the-fediverse/)
2) Your own server's view of the Fediverse doesn't affect the visibility of your accounts to other people. For example my account @FediFollows has 60k followers despite being on a single user server with no relays.
The danger of depending on large instances is you are more at risk of them doing something bad, perhaps accidentally or deliberately.
The surest way to guarantee your control over your account is to own your own server.
News organisations need to think about what their priorities are, and what their accounts are for, before deciding to just sign up somewhere large.
Of course, that's right. I'm spitballing right now: Maybe it would be a good idea for journalists to have an account for "seeing" the fediverse on a big instance, and another one for interacting with it and being "seen" on an instance by their organization (like ours).
Or set up relays to "see" the fediverse but delete the content very rapidly, so to not fill the own server too much. (You don't want stuff from some days ago to reach you, maybe)
@ssundell You're right, I don't like this stuff either. Rest assured that I chose the examples not by accident.
@mho @ssundell It works like a clock that around 9:30 every morning a couple of journalists will call and ask me about whatever the British intelligence slide says that day. So yes, definitely some laziness there.
The list thing has been fixed in the latest version of Mastodon, so now you can actually have lists of people you don’t follow.
@mho @feditips A weakness with relays is that large instances don’t post to them (and if they do, they are going to consume your storage space in no time). So the only way for a post on for example mastodon.social to be searchable for me is if someone on my server is actively following that user, or it’s boosted so my server notices it.
@anderspuck
(… or if it's been retreived by url by some user or bot on your server.)
@mho @feditips
Yes, there are relays based on hashtags which are very handy for themed servers:
@feditips @mho @tchambers Adding relays to a small instance causes an explosion in storage needs (been there, done that, had to remove it).
@jwildeboer @feditips @tchambers
Yes, that's what I'm suspecting. Thanks.
Jan, did you try Fedibuzz relays where the relay has specific hashtags that it follows? That is apparently more careful in resource use than general relays?
More info at https://relay.fedi.buzz/
@feditips @mho @jwildeboer @tchambers
Instead of forcing journalist to have multiple accounts, wouldn't it be easier to have one separate search engine that aims to see all the fediverse? It could even be plugged in directly into the client's interface.
@raphael @feditips @mho @jwildeboer @tchambers
What would it take to build something like that and how would it scale with more mastodon users?
@Zeugs @feditips @mho @jwildeboer @tchambers
Nothing much, really. The fediverse is still relatively small. You can still crawl and index all of it with one (big) commodity server.
@raphael @Zeugs @mho @jwildeboer @tchambers
One server that someone wealthy can purchase and do nasty stuff, the way Google has?
The point of this place is to avoid such pinch points.
@feditips @Zeugs @mho @jwildeboer @tchambers No, one server that is affordable even by an independent software developer (such as myself and my business) and that could offer a service that is almost a commodity.
If any of these service providers tried to go rogue or evil, market forces would peg them down rather quickly.
@raphael @Zeugs @mho @jwildeboer @tchambers
How would it handle opting into indexing? Or would it just index everything it sees?
How would it deal with posts that it cannot see due to restricted visibility?
@feditips @Zeugs @mho @jwildeboer @tchambers
These are implementation details which are orthogonal to the architecture.
Personally, I'm of the opinion that we shouldn't pretend that information in a social network can be private or shielded, and we should educate people that *everything* posted in the internet should be treated as public information.
I know that this is not a popular opinion though, so a search engine could be made opt-in.
@raphael @Zeugs @mho @jwildeboer @tchambers
It feels a bit wrong that people should be sacrificing privacy for the sake of making life easier for journalists. And is it really journalism if it depends on centralised indexing of people's posts?
Couldn't the journalists just have multiple accounts and do some manual digging? It's not that hard to find people, I do it all the time over on @FediFollows
@raphael @Zeugs @mho @jwildeboer @tchambers @FediFollows
Judging from the vast number of news stories that are essentially just embedded Twitter or Instagram posts, it feels like journalists have got used to centralised services and forgotten how they did stuff in the past.
Not all of them, but those that think they cannot do their job without a centralised index.
@feditips @Zeugs @mho @jwildeboer @tchambers @FediFollows there is no privacy being "sacrificed" here, that's my larger point. If the Fediverse ever becomes big enough, *someone* will index it. Having it "opt-in" is like asking "pretty please leave me alone" to 3-letter agencies and Big Data miners. Do you think they'll honor it?
Those that need real privacy should not even be on Mastodon and should use only provably secure communication channels.
@feditips @raphael @mho @jwildeboer @tchambers you can already use privacy. I don't understand the hassle. You can literally browse communities without being logged in. People can agree to robots scraping them. If there is one guy having a search engine someone else can have one too. Having one field for search doesn't mean it's locked to a single search engine. As in Browsers you should be able to select.
@feditips
Yes, #relays could be a solution. Didn't really work out when my private instance discussed it with me. But "topics you're interested in", might be a harder to define for journalists. You want stuff to reach you that's newsworthy, sometimes that happens in #Ukraine, then in #Armenia, then on #Hawaii.
That my visibility is no problem, is good. I know that.
@mho @feditips @tchambers Deswegen gibt es Relais, bei denen man sich anschliessen kann, damit man möglichst ‚viel‘ vom Fediverse zu sehen bekommt.
@mho @feditips @tchambers Ja gut. Klingt logisch. Irgendwie auch tröstlich, dass es bei einem sozialen Netzwerk dazu gehört, sich mit anderen (auf einer Instanz) zu verbinden, statt dass jede.r seine eigene Instanz betreibt und als einsamer Wolf herumsocialisen kann. Es passt in mein Social-Media-Weltbild perfekt 'rein ;)
@mho @feditips @tchambers Ich halte es übrigens für nicht sinnvoll, technisch immer „alles“ sehen zu wollen. Journalismus/Publizismus braucht dann ggf. erweiterte Werkzeuge.
@padeluun Oder eben eine "eigene" Instanz..
@padeluun@digitalcourage.social @mho@social.heise.de @feditips@mstdn.social @tchambers@indieweb.social Hier gibts noch ein paar Infos zu dem Thema:
https://fedi.tips/using-relays-to-quickly-expand-a-servers-view-of-the-fediverse/
@mho @feditips @tchambers sounds like relays are the proper place to handle search. What if a relay offered public search endpoints?
I'm also pretty sure The Vocal Elders of The Fediverse will try to kill that for being a search index. (Been there done that).
@berkes @mho @feditips @tchambers
If the only posts indexed are those by users who opted in for search, I do hope The Vocal Elders would not mind.
@joosteto @mho @feditips @tchambers That in itself greatly reduces the usefullness of a search, though. Opt-in means 1) only those servers who have the features to set that opt-in will participate and 2) only those people who know and care about the feature will be included.
I firmly remain of the opinion that if you **publish** data on the *public* fediverse, you are in no position to demand which clients, apps, and tech people use to consume your data.
Yes, I understand that. But this was debated many times and many, many users see that differently. I think, the different approach is a good compromise.
@mho I think the only "solution" is either to continue this stalemate or someone who doesn't give a rats ass building it anyway.
You see, there's nothing preventing malicious actors from ingesting the fediverse. But there are many ways to prevent goodwilling actors from doing this
@berkes
There were multiple attempts, as far as I remember, and every time the backlash was too big to keep doing it. The only solutions, that were deemed acceptable, were with an opt-in. I think, nobody would want to try it again in the near future
@mho We were one of those attempts with @flockingbird . We gave up, despite hundreds of people cheering for us. Because a handful of people were campaigning (bullying, threatening) just a bit too hard.
I'm sure this will happen several more times, until someone just ignores the bullies and presses through.
@berkes
Ah, I understand. Well, Google already bullies through, doesn't it?
@mho hmm. How is the fact that Google acts like crap, any excuse to allow people on the fediverse to act like crap?
@berkes
You said, that you gave up because of the bullying and I just meant, that there is already someone scraping the data and not waiting for the Fediverse to allow it. Every time someone is forced to give up their project, it is ignored, what Google does. I think, that weakens arguments against such an indexing.
@mho ah. Sorry. I entirely misread (and therefore misunderstood) your comment. Sorry.
But yes, I completely agree. Google (and OpenAi, Yahoo, Yandex) and probably a few hundreds of social-media-data-services are quietly eating up the fediverse that we try so hard to protect.
@berkes
Exactly..
@mho
This is exactly why I've been arguing that there should be a (number of) search index and API-only services which provide this capability to individual servers. Well, that, and the related point that requiring each server to redundantly index the fedi to provide a search for their users is rather wasteful and expensive.
@feditips @tchambers
A year later, search as a service for fedi servers of all sizes is forming up. This is very welcome.
https://www.fediscovery.org/