Day 563: Scraper for Engineers

I met a friend in SF who leads a team that focuses on ingesting their clients’ data to power their product. Much of that work requires web scraping since their clients’ data lives in internal websites rather than as structured data, so she was interested in how Browserflow might be able to help.

After an initial call where we talked about their needs and which parts Browserflow could help with, she sent the website over to her lead engineer who I had a call with yesterday.

Here was his situation:

He’s the person responsible for scraping their customers’ websites
He writes custom solutions using Selenium and spends 40+ hours a month web scraping
He no longer has time to continue building all those connectors and he also doesn’t have time to train everyone else about all the intricacies of web scraping

Given that he seemed to have more scraping work than he could personally handle, he was extremely excited about how Browserflow could help everyone on his team help out with the scraping.

Then he asked his first question: How flexible is Browserflow?

When I told him that Browserflow is extremely customizable and lets you run arbitrary Javascript as well as make API calls, I could see his eyes light up. I was super happy to see that because so much of the work on Browserflow has been about making it flexible and able to handle complex situations that most other tools can’t.

As I continued to answer his questions about what formats Browserflow could export the data as (JSON, CSV, Google Sheets) and how licensing works (for the extension, it’s per user; for the cloud, you can use a single account since it charges by usage), he got visibly more excited because this had the possibility of offloading a ton of hours of work that he could focus on other responsibilities.

When the meeting ended, he said that he’s going to talk to the person in charge of making the purchase decision and that he’s going to push to get Browserflow for his team.

I was stoked to hear and see his enthusiasm, but I’m tempering my expectations for the following reasons:

Enthusiasm doesn’t necessarily lead to a purchase, especially if the person you’re talking to is a nice person and wants to make the other person feel good. This conversation seemed more promising than most especially since there were concrete next steps, but it’s not final until there’s an actual sale.
He hadn’t tried other scraping products in the space. This is an extremely common sign that the problem isn’t that important for the potential customer. He said that the workload has increased significantly more recently so it’s possible that I just happened to show up at exactly the right time — we’ll see! That said, they have been using Selenium and he’s clearly devoted a lot of time using it so it’s clear that this is a problem space they care about; they just haven’t tried looking for better solutions.
He hadn’t tried Browserflow. He had watched the demo video and said he loved it, but it’s one thing to watch someone experienced use a tool and another to try to learn it yourself. It’s possible that he hadn’t found the time to play around with Browserflow given all his work, but I have to remember that his enthusiasm is based on his idea of what Browserflow is rather than his experience with the actual product.

It might seem like I’m being overly pessimistic about what would have appeared to be a fantastic call for anyone, but The Mom Test has been super helpful for identifying common signs for calls that feel promising but may not actually lead to anything.

The takeaway that I had from this call was that positioning Browserflow as a scraper for engineers (as least as the initial hero use case) as I had written about before could be a promising direction:

Engineers are already scraping and understand the problem space
They know how to read documentation and troubleshoot on their own = support is scalable via documentation rather than hiring more support people
Everything else I wrote in my lil love letter about developers