Google Safe Browsing
Google Safe Browsing2020-01-04 15:37:16
I've had a website online for the past 22 years now. During that entire time, I've always distributed my own binaries on my websites.
Unfortunately it seems I'm not able to do that anymore. The reason is due to Google Safe Browsing.
This is a feature from Google that tries to flag unsafe binaries to warn users, which is in and of itself a very admirable goal. Unfortunately there is always a balancing act between preventing bad actors and harming good actors, and in my humble belief, Google has sided way too far toward the latter.
Even though I take software security extremely important, even though my software has always been 100% safe and clean, and even though my domain is very old and has relatively strong authority, it's not enough.
Unfortunately, Google has decided that any newly released binary is suspicious and represents "harmful content" and it recommends "removing it as soon as possible."
The basic theory this falls under is that any software that isn't downloaded enough times is an "uncommon download", and unfortunately I am not a large enough developer to quickly overcome this. For reference, my software receives approximately 1,000 downloads a day. Almost all of these downloads are of higan and bsnes. In my specific case, higan v107 was the software flagged and where this issue first appeared.
My initial thought was that code signing my releases would alleviate this problem. I'm rather opposed on moral grounds to paying $70 a year for a certificate with my legal name on it, or $400 a year for a certificate with my company name on it. Especially because it's a privilege I can afford, but that I know many small and independent developers (especially free software developers) simply cannot afford. Nonetheless, this might be an effort I pursue in the future for the sake of added reassurances.
It seemed as though this might help from a cursory reading of Google's software guidelines, which recommends you to sign your binary releases with bolded text emphasis, but unfortunately after speaking with two Google engineers on this matter, it regrettably will not result in trust being built up with the certificate to bypass this issue going forward.
As an aside, if anyone out there is in a position to help me obtain a Windows code signing certificate (the money is not the issue, I'd pay for it), please reach out to me via my contact page, thank you!
I want to stress that those same engineers gave me repeated reassurances that the warning is safe to ignore, and eventually after some unspecified and secretive number of user downloads (which is in excess of several days for each new software release I'd post), the scary warnings in users' browser download windows (and the much scarier warnings in my Search Console) would eventually go away.
However, at this time, I really don't feel comfortable relying on two engineers and no public statement from Google assuring me that it's a warning that is safe to ignore. The risks are simply too great.
Google controls 92.72% of US search traffic, and 80% of the web browser market. Google Safe Browsing is even used by other major browsers such as Mozilla Firefox.
Put simply: if the warnings were to escalate for not being addressed, I could see my entire website potentially blocked, which is something I've personally seen happen to a friend of mine who also hosted a completely safe binary: EliteMap, which was an editor for Pokemon games.
I've poured the last 14 years of my life into byuu.org, and the software contained and developed here represent my life's work. And I am not going to risk any esclation by attempting workarounds on this site.
Unfortunately, Google holds far too much power as a company. And so unless Google makes changes to this program, I can no longer host binary downloads on my website.
I have set up alternatives to acquire my software for the time being by hosting the binaries on third-party sites. Of course, the same risk applies of them being falsely flagged as "harmful content", but at least the damage is mitigated if things go awry, and alternative plans can be established via my official website.
Right now, I have three alternative distribution strategies in place:
My primary binary distribution channel moving forward is going to be itch.io.
My nightly buildbot by definition also builds official releases, but it doesn't have a very friendly user interface for downloading binaries from.
Nonetheless, it does have the benefit of providing bleeding-edge binaries for folks interested in beta testing and bleeding-edge features.
Discord is a chat service that also allows hosting downloadable files. It can serve as a fallback in the unlikely event my binaries are ever completely blocked in browsers, by running the standalone Discord application.
I'll be posting my release binaries going forward on my Discord server's
Permalink • 8 Comments
User2020-01-04 18:25:35Try to move downloads to a subdomain.
byuu2020-01-04 18:41:27That wouldn't work. Having a direct link to the binaries, no matter where they are, is the risk.
User2020-01-04 21:54:41Sounds like a good time to support Firefox. ;)
nitro2k012020-01-05 02:05:42Have you considered using Github releases for downloads?
byuu2020-01-05 04:30:49User, Firefox uses Safe Browsing as well.
nitro2k01, that would have exactly the same problem. GitHub is a very important site for me as well.
nitro2k012020-01-05 15:03:33How would hosting the binaries on Github have the same problem? Github is hosting literally millions of "uncommon downloads" of binaries in releases from various repos. It's part of their core business model. If this was a problem for Github, the site would be unusable because most repos would be unindexed by Google, or users banned left and right simply for distributing less common binaries. Of course this is not the case and presumably Github is on a Google whitelist because they respond to takedown and probably actually investigate whether a binary is malicious or not.
6t8k2020-01-05 19:21:19Reading your most recent article on Medium;
"ZIP files leak their .exe filenames even when encrypted, so it would have to be 7-zip which is less commonly available";
You can still use zip by circumventing the that the .zip format is leaking that metadata by zipping the .exe, and zipping that file again using a password.
Then provide the .txt file as a separate download link (to stay at two layes of un-zipping the user has to do).
But yes, that's ugly and would introduce an extra barrier the consequences of which may be unbearable.
Trying to outright hide the download from the crawler would probably do no good long term, I agree.
But I wouldn't really name the .zip approach in the same breath, because from an outsider perspective, what is hosted and downloaded by the user is random data. If we've come as far as disallowing the exchange of random data, then we have a grave problem (keyword: net neutrality etc.) and I'd firmly oppose to that instead of giving in to corporate interests, stifling FOSS developers/free communication.
byuu2020-01-06 09:42:39nitro2k01, because it only takes down the specific repositories targeted, rather than the entire site in those cases. No such luck when the target was kafuka.org.