T here are some retailers with physical presence I truly enjoy shopping from. Perusing the aisles of Home Depot is always fun and a good way to start a DIY project you didn’t know you needed to start. Playing with cooking gadgets and shiny pots & pans at Williams-Sonoma is a great way to quickly run out of counter-space. The workout clothes at Lululemon are buttery soft. And the knowledgeable people working at REI are always there to help fit a hiking backpack properly.

But sometimes, I just need to order something online, and it’s an uphill battle. Some sites have made it extremely hard to shop from them by using overzealous, antiquated “bot detection” that offers no recourse for humans. The biggest offender that frequently blocks my daily driver, an Ubuntu desktop, is Akamai. They usually block the search and product pages, even if I’m logged in. This also means that they block me if I click through via Google, via a Google Ad or via an email newsletter/spam they sent me. So if I’m really determined to buy something from them, I need to do one of two things:

1. Pull out my laptop.
2. Use Firefox’s built-in fingerprint resistance. Go to about:config and turn privacy.resistFingerprinting ON, clear all cookies, visit the site.

The fact that the latter works is particularly maddening if you consider that I can then revert privacy.resistFingerprinting to OFF, and then I’m still cool to browse like a human. It seems like Akamai sets a long-lived cookie that says I’m indeed not a bot, and I’m not sure how often it is updated. The heavy reliance on browser fingerprinting seems to not be particularly effective against bots but very effective against me ordering more spoons¹.

But what does resist fingerprinting do exactly? The kitchen sink, as documented on the Mozilla site. Why? Let’s try to decipher what Akamai’s bot detection does when loading a page. There are two hefty and obfuscated scripts served from the retailer’s domain. Loading them in an isolated <iframe>, and with the help of AI, I monkey-patched² as many browser JS calls as I could think of. Then we can see that (at a minimum³), the following pieces of information are requested:

• from the navigator, grabs userAgent, platform, appVersion, appCodeName, language, languages, hardwareConcurrency, deviceMemory, maxTouchPoints, webdriver (automation check), pdfViewerEnabled, mediaSession.
• navigator.userAgentData.getHighEntropyValues()
• navigator.plugins and mimeTypes.
• navigator.connection
• navigator.storage.estimate(), likely to gauge if it’s a fresh and/or private browsing session
• navigator.permissions
• Notification.permission
• navigator.mediaDevices.enumerateDevices()
• performance.memory
• performance.getEntriesByType()
• performance.now()
• screen object: width, height, availTop, colorDepth, pixelDepth, orientation.type.
• devicePixelRatio
• getComputedStyle()
• document.fonts
• Intl.DateTimeFormat().resolvedOptions()
• speechSynthesis.getVoices()
• Calls to 2D canvas functions like fillText, arc, bezierCurveTo, and then the rendered output is hashed
• A ton of calls to Function.prototype.toString and Object.getOwnPropertyDescriptor, probably to detect if it’s been watched by inspecting signatures of native functions
• getContext("webgl") and WEBGL_debug_renderer_info to get GPU vendor and renderer.

Through a process of elimination, I discovered that it’s the last call that was decisive in tripping the bot false positive. When browsing on my desktop, I use the CPU renderer (llvmpipe) to keep my lowly GPU available for my CUDA code. Fine, a red flag indeed, but just setting webgl.enable-debug-renderer-info to false lets me through, no privacy.resistFingerprinting needed. An easily hidden value was enough to prevent Home Depot from selling me that electric lawn mower.

So what can be done?

Step one, and the most important one: Teams that are considering such solutions should first quantify how big of a bot problem they have, if any, what kind of bot problem, and what it’s costing the business. Then they should pit all vendors against each other, in a carefully designed experiment with predetermined acceptance criteria and trade-offs, i.e. acceptable session drop, latency increase, error increase, GMV drop guardrail, etc. If you don’t have a way to run experiments, then at a minimum request raw logs of all hits on the edge, accepted or rejected, and correlate it with your own logs offline.

Experiments
Having a way to easily run, track, and analyze experiments is table stakes. Without a culture of experimentation you cannot have true data-driven decision-making.

Second, security vendors as well as their customers should offer a “Think this is wrong?” button as a way for a human to report that they are being unfairly blocked. That page may even have a modest Proof of Work deterrent, and the data can be analyzed offline daily as a way to monitor the effectiveness of a bot protection solution. Along the same lines, keeping a tiny holdout as a perpetually running experiment is not the worst idea.

Third, the tech needs to continuously improve. I don’t love the privacy implications of browser fingerprinting, but everyone is using it, and it’s relatively easy to evade.

Until then, if someone can convince my wife that I deserve (at-least-one) RTX Pro 6000 for Christmas so I can browse with hardware acceleration turned on and run experiments on my GPU, please reach out!