Essays by Erik

Where to Put the Bolt

Wed, 20 May 2026 00:00:00 +0000

From time to time, direct reports will voice concerns about their output. This is typically some variant of how much code they’ve not written recently while coordinating some complex aspect of a project. Recently due to leadership emphasis on “AI fluency” this is with respect to token usage.

My goto reply is the following parable. First told to me by a manager early in my career, now others ~~suffer~~ receive it similarly.

A factory worker retires after 40 years on the job. The next week a critical machine breaks down bringing the assembly line to a halt. The worker was an expert in this machine.

The company tries everything to get it running. They take it apart and put it back together, bring in consultants, fly in workers from other plants. Nothing works.

Finally, they contact the worker, “Please help us get this machine back online. You can charge us whatever you want.”

Factory worker comes onsite, walks around, finds a bolt on the floor, screws the bolt back onto the machine, and presses the power button. It whirs to life. Worker tells them, “All set. I’ll invoice you.”

The worker sends in the invoice and the factory head calls them up, “What do you mean $20K?! You spent 20 seconds on this. I need this itemized.”

The worker sends in an itemized invoice:

Bolt: Free

Knowing where to put the bolt: $20K

Outcomes »> outputs. 🔩

The (AI) Road Ahead

Fri, 24 Apr 2026 00:00:00 +0000

The job market does not have to be net negative due to AI.

As I’ve confessed before, my 🔮 is cloudy. With that profession, here are two theories with respect to the job market and this moment of AI uncertainty.

I believe these hold for at least the conditions where superhuman intelligence is not achieved. Otherwise, we will have unstable equilibrium.

Crushing the Competition

Companies are marveling at AI in the moment. They say “Wow, look at what we can do with this. We can be so efficient.”

There are two roads to take from there.

The organization continues, saying, “We can do the same amount of work with fewer people.”

This is the scarcity mindset. The path Block has taken, laying off 40% of the company.

we’re not making this decision because we’re in trouble. our business is strong. gross profit continues to grow, we continue to serve more and more customers, and profitability is improving. but something has changed. we’re already seeing that the intelligence tools we’re creating and using, paired with smaller and flatter teams, are enabling a new way of working which fundamentally changes what it means to build and run a company. and that’s accelerating rapidly.

– Jack Dorsey, CEO

In the other, the organization instead says, “We can do so much more with the same amount of people.”

This is the growth mindset. These organizations keep their existing hiring plans or expand them. They retool and reorganize. They accelerate, not maintain, their roadmaps. They pull ahead of the competition, the ones maintaining their existing pace with fewer people. They crush them.

The job market does not have to be net negative due to AI.

Computers as Jobs

Computer used to be a job that people did. Very intelligent people. Often women. We’ll refer to them as capital-C “Computers”.

These Computers, well, they computed complex maths: planetary orbits, rocket trajectories, flight paths.

Then computers as we know them came. Computers that could work tirelessly, consistently, and with near flawless execution given proper instructions and oversight.

What about the Computers? Were they relegated to the dustbin of economic history? No.

Yes, there may have been momentary disruption in how we applied them.

Capitalism is ruthlessly efficient.
It would be economically wasteful to not utilize these individuals.
It would be capitalistically irrational.

And so we found new uses for them. In overseeing the computers. Taking on higher order levels of work. New abstractions. The job market was not zero sum, exchanging Computers for computers. Computers are the spiritual ancestors of today’s software engineers.

And so it can be with AI.

While handcrafting code may become a hobbyist endeavor, utilizing their human intelligence will not.

We will find new, knowledge-based uses for intelligent people. The uses are hard to see right now in the fog of war.

Capitalism is ruthlessly efficient.
It would be economically wasteful to not utilize these individuals.
It would be capitalistically irrational.

The job market does not have to be net negative due to AI.

Winter Is Coming

Fri, 10 Apr 2026 00:00:00 +0000

Winter is coming.

– House of Stark

A year ago, most engineers, myself included, thought Copilot and Cursor were “cute”. That’s nice. They can write tests or auto-complete a few lines.

Today engineers turn over entire features to Claude Code and let it burn.

A sea change has occurred in AI capabilities these past 6 months.

Security is not unaffected. A few recent Anthropic announcements:

…found and validated more than 500 high-severity vulnerabilities.

– Evaluating and mitigating the growing risk of LLM-discovered 0-days

…discovered 22 vulnerabilities over the course of two weeks…a fifth of all high-severity Firefox vulnerabilities that were remediated in 2025.

Partnering with Mozilla to improve Firefox’s security

…capable of identifying and then exploiting zero-day vulnerabilities in every major operating system…subtle or difficult to detect…27-year-old bug in OpenBSD—an operating system known primarily for its security.

Assessing Claude Mythos Preview’s cybersecurity capabilities

Where does this leave us? My 🔮 is cloudy, but here is a guess.

Micro-pessimism

The next 2-3 years will be bad. It will feel like a reset of the internet to the early 00’s when OWASP Top 10 in the wild was the norm, not a cautionary tail.

We’ll see an unprecedented volume of high-severity vulnerabilities in open source projects foundational to modern software. Teams will be buried in triage and patching.

Adversaries will abuse frontier models to identify and exploit new vulnerabilities in open source, compromise supply chains, and directly attack corporate applications.

There are tremendous asymmetries for attackers. They can scale up more agents, hunting across companies and attack surfaces. In spite of organizations’ best defenses, failure only requires one successful attack. And of course, attackers have an asymmetry in morals.

Given this, there are only a few obviously useful asymmetries in the other direction.

First, organizations know their most worrisome areas (weakest || most important). They know what keeps them up.

Second, organizations are the only ones with access to their codebase.

Organizations should act on both in combination. Run agentic vulnerability scanners against codebases. Direct them to the most worrisome areas and entry points first. Prioritize ruthlessly. Repeat.

Because attackers can’t–yet. They’re limited to external attack surfaces.

Go.

Macro-optimism

The future can be better. The systems we build and code we write 1-2 years from now, reviewed by frontier models for flaws from the start, can be the most secure in history.

5 years from now, we can shake out the worst of the legacy issues. The new norm can be laughably secure systems.

Onward.

The Weight of Privacy

Thu, 30 May 2024 00:00:00 +0000

N.B. The goal of examples and cases below is to examine what went wrong, not cast aspersions at companies or past decisions.

What Is Privacy?

Video: Steve Jobs On Privacy (34s)

Privacy is the right to control your data. Boil away the legal details and this is its essence.

This post provides a survey of real privacy issues, and how they can be mitigated. The goal is to convey the weight of privacy that we all bear as engineers handling user data.

Why Care about Privacy?

Video: Mythic Quest Has Been Hacked (42s)

Respecting privacy is users first.

Set aside the alphabet soup (GDPR, CCPA) of privacy legislation and how you may feel about these laws. Many, many users care deeply about how their and their customers’ data is used; a significant degree consider privacy to be a fundamental human right. Not sufficiently considering user privacy in your products and systems is to disrespect your users’ privacy, violate their rights, and go against company values to put users first.

When users discover violations, trust is lost. Users expect that by default:

Products protect privacy (Public By Default: What Venmo and the Whole World Knows About You)
Data access is limited (Tesla workers shared sensitive images)
Internal tools are safe (Uber Allegedly Stalked Users, Snapchat Employees Abused Data Access)

This is trust that takes years to build, but would take seconds to break, and may take forever to repair.

Privacy Engineering (and How It Is Different from Security Engineering)

Privacy engineering teams builds frameworks to meet the privacy expectations of users. Security and privacy are adjacent but distinct disciplines. Privacy is about the right to control how personal data is collected and used. Security is about protecting that data, including preventing unauthorized access, data corruption, or theft. Often good practices overlap between the two.

The first OWASP Top 10 dropped in 2003 with your favorite PHP and IIS-powered site vulnerable to script kiddies. Twenty years later robust security protections come out-of-the-box in popular frameworks. For example, a default Rails app provides protections for CSRF, SQL injection, and more.

Privacy engineering is today where web app security was 20 years ago. There are few paved paths and even fewer plug-and-play frameworks for achieving the ends expected by users and demanded by legislation. Because of this, the frameworks companies use to ensure they respect user privacy evolve drastically year-to-year.

By seeing the consequences of privacy issues in practice, you can better understand why you’re asked to implement privacy controls or adopt privacy frameworks.

Selected Case Studies and Lessons

Invert, always invert.

— Charlie Munger

The front page test is an excellent litmus test for privacy issues. It’s useful to examine real-world privacy issues and then invert these into practices to adopt instead.

In 2011, Google agreed to an FTC consent decree requiring 20 years of independent privacy audits. This came from Google Buzz sign-up and data sharing. In the FTC’s own words:

Google led Gmail users to believe that they could choose whether or not they wanted to join the network, [but] the options for declining or leaving the social network were ineffective…the controls for limiting the sharing of their personal information were confusing and difficult to find[.] [Consumers were] concerned about public disclosure of their email contacts which included, in some cases, ex-spouses, patients, students, employers, or competitors.

In another case in 2022, CNIL–France’s privacy regulator–fined Google and Facebook €150M and €60M respectively for nudging users to “accept all” cookies in their consent banners. Consent must be provided freely. That means making it as easy to reject non-essential cookies as accept them.

In 2022, a Twitter user called out Stripe for a checkbox to subscribe to email product updates. The issue was quickly remediated once flagged.

Users have high standards for companies and expect the same excellence and clarity in the privacy experience as the rest of the product experience.

Inversion

Privacy means people know what they are signing up for in plain English…Ask them.

— Steve Jobs

When building user journeys–for example, signup pages–Jobs’ advice from the first video is trustworthy.

Ask users unambiguously for permission. A user is signing up for a product? Ask them. Want to collect web analytics? Ask them. And if asking is difficult for whatever reason, ask your privacy-focused colleagues whether there are alternatives.

Respecting the Global Privacy Control setting for all users is a clear illustration of prioritizing and respecting users’ privacy. Beyond merely complying with legal requirements, you can prioritize users’ privacy preferences. Even in jurisdictions where it’s not required, respect users’ GPC setting opting them out of advertising cookies. Users with the GPC setting proactively communicate their preference, so respect this and don’t set those cookies or use their data for that purpose.

Did the reason for collecting the data remain consistent?

In 2022 as part of a 20 year consent decree, the FTC ordered Twitter to pay $150M for collecting phone numbers for 2FA and then using those numbers for targeted ads. The data used was not clearly annotated, leading to this mishap. Users were frustrated. They provided their phone numbers with the understanding that they’d protect their accounts, not enable advertising.

A 2020 blog post was picked up by Hacker News with concerns over Stripe’s collection of advanced fraud signals. There were concerns that Stripe’s privacy policy allowed for using this data for advertising purposes. Patrick Collison, Stripe’s CEO, quickly promised that this data is collected exclusively for anti-fraud purposes. Stripe followed this up with a blog post and updates to its privacy policy.

Distrust of how companies use data is the norm. Regardless of your intent, users are quick to point out incongruencies and potential loopholes between policies and practices.

Inversion

You must adhere to the purposes of processing for which data was collected. In practice this is called business purpose limitation. If data was collected for anti-fraud purposes then that’s what the data can be used for. You can’t later change your mind without gathering consent again.

Individuals have the right to a copy of their data (via a Data Subject Access Request), often grouped by the purposes of processing. Imagine seeing data a user provided for risk purposes displayed under analytics because it was copied inappropriately from one system to another. Users also have the right to be forgotten and for their data to be deleted.

Data annotations are key to knowing what data you have, to whom it belongs, and why it was collected. Annotations are foundational to infrastructure limiting access to closed account data, a special case of business purpose limitation. They can also be used to identify data to delete when you no longer have a justification for retaining it.

When creating or editing data models, add annotations. Yes, this should be the case even if the dataset is intended to be short-lived or only go to QA. Usage evolves, data is forgotten, people move on. Without annotations you can’t account for the data you have.

While privacy engineering teams can create inference models and tools to improve the ease of annotation and accuracy, you are ultimately the expert in your data and responsible for us handling it well.

Is access to data limited to proper use cases?

In 2024, a Carta employee inappropriately accessed and shared customer data to generate interest in an unauthorized, secondary sale of stock. As a result and to stave off brand damage, within 72 hours Carta shuttered their entire secondary stock sale business.

One VC [said] it was akin to ‘Oracle using your database to share supply-chain data or Salesforce using your CRM data to inquire about the state of your sales leads.’

Abuse of internal tools and data access is deeply corrosive to user trust.

Inversion

Least privilege dictates that only the people and workloads that need access to specific data for their job have it. That access must also be monitored and audited.

Authorization controls including two-person confirmation and expiring permissions on internal tools and data can enforce least privilege. These checks may add friction to your workflow. And, yes, you are trying to help your users. Remember though that the abuses mentioned at the beginning of this article stemmed from permissive access. Continual investments in ergonomics can minimize the impact of these controls on daily productivity.

In the long term, privacy preserving technologies like tokenization, pseudonymization, and differential privacy can enable open access to data without compromising an individual’s privacy.

A Vector, Not a Destination

Like security or reliability, privacy is a vector, not a destination. There will not come a time when the work is done.

In all likelihood, your company’s use of data is complex and building excellent privacy controls means that privacy engineering teams must make it simple for engineers to reason about its collection and use.

While building your own products and systems, reflect on the above cases. Whether you are using a privacy engineering framework or implementing controls on your own, we all bear the weighty responsibility of respecting user data and privacy.

The RSU Cookie Jar

Mon, 13 Nov 2023 00:00:00 +0000

Who stole the cookie from the cookie jar?

— Sesame Street

If a company’s valuation increases and its growth prospects taper enough, they’ll switch from granting employees Incentive Stock Options (ISOs) to Restricted Stock Units (RSUs).

With RSUs, the company essentially gives the employee company stock over a period of time.

Double-Trigger RSUs

There are different triggers on the RSUs that must be satisfied before the employee receives units of company stock. A time-based trigger is nearly always present on the RSUs. This is the same as ISO vesting where an employee must achieve tenure at a company. RSU grants typically have a one year cliff when an employee first joins the company; subsequent grants then vest quarterly.

Sometimes there’s a second trigger on private company stock where not only must the employee be at the company a period of time before they receive the stock but the company must also experience a liquidity event like a public offering or an acquisition.

Until both triggers are met, employees aren’t taxed by the government on their vested RSUs. In exchange, the IRS requires that there be a substantial risk of forfeiture. Often this risk comes in the form of expiration.

Susan’s RSUs

Let’s go back to Susan. Widgets.io no longer grants stock options and instead has switched to RSUs. This happens when the future expected growth of a stock is much lower. If the value of a share is $1,000, it’s far less likely to 100x again. So companies switch to RSU grants to incentivize employees joining.

When Susan joined Widgets.io 6 years ago, Susan received a grant for 1,000 double-trigger RSUs vesting over 4 years. 7 years later that initial grant is fully vested and the company shares are worth $2,000 each. But the bad news is that Widget.io is not planning on going public or being acquired anytime soon. And the worse news is that the RSU plan is written such that grants will expire after 8 years. So on paper Susan theoretically has $2M of stock that may just vanish in a year if the second trigger isn’t met.

The company wants to retain Susan. If they start letting RSUs expire before the second-trigger is met and they’re converted to full shares, employees will lose trust in their compensation.

But the IRS requires that RSUs carry substantial risk of forfeiture. What the company can’t do is say “Don’t worry, Susan, even if your $2M in RSUs expire we’ll just give you $2M immediately in new ones that expire in 8 more years.” A statement or practice like that would immediately throw up flags with the IRS.

The company would be signaling there isn’t substantial risk of forfeiture. And if the IRS noticed, not only would they say that Susan owes taxes on the $2M because the company winked and nodded about her shares expiring but every employee with RSUs would owe taxes once their time-based trigger was met.

Another way to think of it is like a giant, hermetically sealed cookie jar. The cookies in the jar represent the RSUs belonging to each and every employee. So long as the jar stays sealed, the IRS doesn’t tax employees.

Some of the cookies though are due to expire, and those employees are sad. The dilemma is that if just one person were to extract their cookies or have their expired cookies replenished with a new one, the IRS consider the seal broken. Now all those employees are required to pay the cookie tax.

The company is responsible for withholding income tax from the RSUs that just failed to meet their risk forfeiture test. This could be problematic for a company with billions of dollars of employee income tax liability. Most non-public companies don’t store those sort of cash reserves on hand for the purpose of paying employee taxes.

So what’s a company to do?

Stripe’s Fundraise

Stripe faced this conundrum in early 2023. The company was about a year out from their first RSU expiration. Employees were restless to know what the company would do since the economy and market didn’t seem to indicate that a public offering would be imminent. Would the company allow those first RSUs to expire? This would signal to remaining employees that the company may allow their RSUs, even with distant expirations, to become worthless too.

Stripe in this case decided to raise a substantial sum of money. The fundraise was done in such a way as to satisfy the liquidity trigger requirement. In addition, enough funds were raised so that employees could sell as much of their now vested RSUs as they desired.

The exact mechanism of meeting the liquidity requirement isn’t clear. A brief, circular, and hand-wavy explanation was once provided along the lines of “It’s somewhat complicated, but I’ll just say that it was met.”

Typical liquidity events are public offerings or acquisitions. I imagine the legal definition of an acquisition involves a transfer of control. Maybe on paper Stripe was “acquired” via a transfer of control to a shell company. I’m sure outside counsel earned their stripes (no pun intended).

22% vs 37%

In Stripe’s fundraise, they gave employees a choice of how much withholding they would observe. In order to simplify the administration, if you had <$1M in RSUs vesting you could choose for the company to withhold either 22% or 37% in federal taxes.

22% was the minimum withholding the IRS requires below $1M; 37% was the highest tax bracket at the time.

Employees now faced another choice. They were told they’d be able to tender (sell) as much of their stock as they wanted. The tax event and the tender offer were separated by over a month. Because of this, I felt there was some risk. What if the company hit the liquidity event and had cash for the taxes, but some adverse event (for example, a breach) occurred that caused the tender offer to be withdrawn before employees could sell their shares.

In addition, Stripe would apply this same withholding rate to all future RSU vesting events for the year. And no further employee tender events were planned.

Lest I end up in another tax man pickle where I chose 22% withholding but actually owed more and couldn’t sell, I opted for the conservative route of 37%.

The added benefit of this is that with a federal marginal tax rate <37%, you are actually guaranteed a refund on your year end taxes. This essentially acts as tendering a small portion of your stock without actually being at the mercy of the company to plan a full tender event.

In the end, I tendered all my available shares. After already having seen the share value drop > 50% during my tenure, I preferred to index into the broader market. With more illiquid shares vesting over the next year; it was the right psychological move for me. If the price went down further, I’d feel good about selling. And if the price went up, I’d experience the gain on the future vests.

Win win.

Of ISOs and Edge Cases

Wed, 25 Oct 2023 00:00:00 +0000

It’s not fun…taxes…it’s the worst part of the game!

– Donny

Startup employees often receive stock options. These are most commonly Incentive Stock Options (ISOs), which carry special tax treatment.

When a company has a positive financial result, its employees are exposed to some peculiar edge cases within the US tax code. Below are some interesting (and sometimes simplified) scenarios.

Stock Option Background

Susan is an employee of Acme.ai. She is granted 1,000 Incentive Stock Options with a strike price of $1.00 vesting over 4 years.

Contrary to what many people believe, the employee isn’t given Acme.ai stock. What Susan receives (is granted) is the option to purchase the company’s stock at a predefined amount. Hence the phrase stock option.

Employees can only purchase this stock as they gain tenure at the company. This is referred to as vesting. Vesting follows a schedule. Most commonly the stock vests over 4 years. The first 25% vests at the employee’s 1 year anniversary. Then 1/48th vests every month thereafter.

In the above case, Susan can buy 1,000 shares of Acme’s stock for $1.00/share.

Why would she do this? Well, she may believe that the stock will someday be worth more than that.

The cool thing, is that Susan doesn’t have to buy it as soon as it vests. She may be equivocal about the company’s future success, especially in those early years.

Instead, Susan can work for Acme.ai for 6 years and decide to purchase those 1,000 shares after that time. Maybe Acme.ai is then valued at $1,000/share. That would be $999K of profit. This is a pretty great deal. Susan gets to buy the stock at the price she was offered 6 years ago and gets the value it’s at today.

It looks like Susan’s hard work for Acme.ai was worth all those late nights and ramen.

Alternative Minimum Tax

Hers’s where things start to get wonky. After 6 years, Susan is ready to move from Acme.ai to Widgets.io. Her stock options usually require that she exercise (buy) any shares within 30 days of leaving Acme.ai. $1,000 is meaningful to Susan but she believes in Acme.ai’s future.

So she buys all 1,000 shares. Now the weird thing is that, yes, Susan owns shares of Acme.ai that are technically worth $1,000,000. However, it’s all funny money. The startup stock is illiquid. She can’t sell it to someone else. She knows of no future IPO or acquisition of Acme.ai. It’s all just Monopoly money.

But in the eyes of the IRS, Susan may be taxed on that funny money. The IRS will require that Susan complete forms considering her Alternative Minimum Tax, a parallel tax code that was created to ensure that high earners with uncommon income situations pay their dues.

Through the infinite wisdom of some tax code creating lawyer, the gains from Susan’s stock are treated as income under the Alternative Minimum Tax code. There are some conditions that attenuate the impact, but overall Susan must think carefully about leaving Acme.ai or exercising her options lest she end up with a tax bill she can’t afford on that $1M of funny money.

Worse still Susan may choose the exercise all 1,000 options, leave for Widgets, and pay a large tax bill that year. Later if Acme goes bust and her shares are worth $0, she doesn’t get a refund for that tax on her paper wealth. She would get to claim a $999K capital loss, but that probably does her little good unless she has other gains for it offset.

Deal Close

Now, let’s say Acme.ai signed a letter of intent to be acquired by MegaCorp. Susan may have the opportunity before the deal closes to buy her shares for $1,000 and receive $1M from the deal broker after close. If she doesn’t, the deal may be structured whereupon close MegaCorp will net exercise her shares and just give her the net proceeds through payroll.

Why purchase the shares early? Well, if MegaCorp processes the proceeds through payroll, Susan will also be subject to Medicare-related taxes. At her income level this will be 2.25% of additional taxes.

So should Susan purchase her shares pre-close? Maybe. It would certainly be nice not to pay an additional $20K in taxes. However, deals fall apart all the time. Susan is confronted with a tax pickle choice.

She could steal away (legally) from 1st base and avoid the Medicare taxes by exercising pre-close. However, if the deal falls apart she would then owe AMT as described above on the $1M of funny money income for which she no longer is receiving cash payment. Susan may effectively go bankrupt in this case for lack of ability to pay the IRS.

Will she make it to base safe?

The above is a real scenario. I knew employees during the acquisition of Duo Security who faced this very situation. They chose the extra Medicare taxes–likely totaling over $100K–over the risk of bankruptcy.

83(b) Elections

Some companies offer their employees the chance to early exercise their options. In Susan’s case this would mean that she could pay $1,000 some time after she starts at Acme.ai and before the stock has actually vested. Then as she gains tenure with the company, she automatically receives the stock she has already purchased.

The nice thing here is that the IRS lets her claim the stock as income immediately. And because Susan is buying stock that is worth $1.00/share for $1.00/share her net gain is $0. So there are no tax implications.

Six years later when Susan’s stock is worth $1,000/share, she already “paid” ordinary income on the stock option exercise and all of her gains are treated as long-term capital gains taxed at 20%.

But (yes, there’s a but) this is only the case if Susan remembers to fill out and file an 83(b) election with the IRS stating that she wants this specific tax treatment. And she is under fairly strict time requirements to file this soon after early exercising the shares.

If she doesn’t, the IRS will tax her as she vests her shares. So if at year 4 the stock is worth $900/share, the IRS may tax her newly vested shares under the Alternative Minimum Tax as if she made $899/share on every share she vested.

Like most tax-related niceties, early exercises are a fun feature of the tax code so long as you understand the paperwork requirements.

Qualified Small Business Stock

Another cool feature of early exercises is the Qualified Small Business Stock. If you early exercise and file your 83(b), you start the clock on avoiding Federal taxes.

Essentially for Susan, she needs to hold the stock for at least 5 years before acquisition, and Acme.ai needs to have had <$50M in assets when she exercised.

If these conditions are met, then Susan doesn’t pay Federal taxes on the greater of $10M or 10 times her basis.

In Susan’s case, she would avoid Federal taxes on her $999K of profit.

Technically the QSBS benefit doesn’t require early exercise. But startups that grow past N years of life tend to raise capital that causes the company assets to exceed $50M and disqualify it from the benefit before employees can exercise most of their stock. Early exercise gets you in before the company is disqualified.

All in all, ISOs are largely a positive feature of the tax code. Their pointy edges though have maimed more than a handful of employees over time.

The Sacred Timeline

Mon, 25 Sep 2023 00:00:00 +0000

Humans are remarkably great at revisionist history. Business is no exception. When I discuss a startup or growth company’s future prospects, my crystal ball is typically cloudy. And it’s cloudier the closer I am to the operations and execution of the organization. It’s easier to prognosticate positively about a competitor when you only see their good press and growth rates from the outside.

Patrick Collison once said at a Stripe all-hands, “We live our self-doubt.”

On the inside, the likelihood of negative events feels higher than positive ones. This is probably because negative events occur easily through inertia and complacency while positive breaks require tremendous effort to inflect the business.

Looking forward, it feels like we as employees are the Time Variance Authority working to ensure that the timeline of the company stays on the best, predictable path. We work to cleave branches that threaten to destabilize our trajectory.

Retrospectively, the timeline followed feels self-evident. All the branches that could have been are cleaved and the narrative is set. Of course we ended up here, that’s where we are.

This comes up from time-to-time with former coworkers from Duo Security. Cisco acquired Duo in 2018 for $2.35B. This was undeniably a great result.

One of the former Duo executives likes to quip along the lines of “Yes, and we could have done a lot better if we had been more disciplined.”

I’m not so sure about that. It’s hard to know prospectively which ideas may lead to the next $100M in ARR. And I’ve been skeptical of enough ideas that do so to remain skeptical of my own predictive abilities.

Conversely, there were a lot of paths that might have been worse than the result Duo arrived at. Nation state adversaries were constantly trying to breach Duo and compromise customers’ authentication systems. Attacker luck or a simple error like an employee getting phished may have led to this. For a security company whose sole purpose is protecting customers, that’s game over and would decimate the company’s value.

About a year before the Cisco acquisition Duo was in the process of being acquired by another MegaCorp for just under $1B. A signed term sheet and everything. Just before close, the acquirer pulled out. It felt devastating at the time, like we’d peaked. Duo instead raised $70M and just 12 months later was acquired for >2x those terms. Not a bad ROI for a year’s time.

There’s a reason that one of Stripe’s Operating Principles was once: We Haven’t Won Yet.

It’s easy to revise the narrative of what occurred; fortune telling forward is far more fraught with peril.

Avoid complacency and stay diligent.

More Strange Things

Sat, 04 Feb 2023 00:00:00 +0000

More strange experiences, this time less corporate.

You’re a Hard Man to Find

When I was in undergrad, my brother or neighbor called me from the city where my parents lived in another state. They said that the police were looking for me. Not really the phone call I wanted to get.

I called the number that they. It turns out it was the police local to me who wanted to get ahold of me. The somehow looked up me up via my parents. Umm…

I called the number and a detective answers. He asked me if when I last saw my car and what state it was in. I told him I saw it that morning when I left my place and biked to class. Wrong answer. Apparently, I fabricated that memory.

In the middle of the night the wheels on my car had been stolen. A neighbor had seen another truck in the parking lot and a bunch of wheels in the bed. My truck was left on the ground. They grabbed the plate number and reported it to the police. Local SWAT suited up and enjoyed some nice practice breaking down their door and recovering the wheels. The detective told me that they had had fun getting them back.

I could head downtown now and recover the wheels from evidence. Problem was that I didn’t have a car to get them. What was I going to do, roll them home one at a time? No luck on getting an assist from the department bringing them to me. Luckily a friend helped me out.

The detective though mentioned to me that I had been hard to track down. I didn’t have a presence on social media and had removed my contact info and name from the university’s public directory.

You’re a hard man to find, Mr. Barbara. Please don’t ever become a fugitive of the law.

Mexico Bus Robbery

In high school, I did a multi-week summer langauge immersion in Mexico. Every day I’d take the bus to the language school from my host house. Let’s call it the #2 route. Usually we’d take another bus back.

One day late in the evening another student and friend of mine who was staying in the same host house wasn’t feeling too great. We were ready to head home for the evening. The #2 bus going the opposite direction was pulling up, and we were impatient. Brilliant idea, we’ll just take that. It probably takes the same route in reverse. Clearly, I didn’t understand bus routes.

We got on and rode for a bit. It kept going and going and going. Soon nothing was recognizable. It was getting dark. More and more passengers were exiting the bus and it didn’t seem to be heading at all in the direction we needed.

Finally we were down to just 7 or so people on the bus including the driver. Two men got up from their spots. One went to the front and the other toward the back where the remaining passengers were located. After hassling the riders in front of us, he approached us with his hand in the pocket of his jacket and what looked like a solid object pointed toward us.

Give me all your money.

My friend generously replied.

I don’t have anything, he’s got it all. (pointing to me)

That was the most fluent and fastest Spanish I’d ever heard him utter. This was also a lie. It was nearly the opposite. I had lost my wallet on the trip and only had Mex$110 ($11 at the time) left: a 100 peso bill and a 10 peso one. My friend on the other hand had a wallet full of USD, cards, etc.

I reached into my pocket and grabbed one of the bills, hoping it was the 10 peso note. Bad luck.

After handing him the bill, he was quite unsatisfied and started patting my cargo shorts. Yes, it was the 2000’s. I figured it would be bad to hedge at this point and give up the other bill, so I offered him the laundry detergent I had purchased earlier that day. He was clearly disappointed and went away.

Up front I could see the other guy clearing out the driver’s change sorter into a bag. He motioned for the driver to pull over. Before they stepped off, he ripped the stereo receiver out of the bus and said something in Spanish about the driver’s mom.

At that point we hoped it was the end and we’d now get to return home safely. The driver instead went a few blocks down, pulled over, and told the rest of us to get off. So much for solidarity.

We slinked through the shadows, two gringos running between street lights until we found an open cantina that would let us use their phone to call a cab. It took 2 or 3 bars until we found one. Then we had to convince them to take USD, which was less difficult than I imagined. Finally, we made it back to our host house and collapsed on the entry steps tired from the subsiding adrenaline.

We didn’t mention it to our host parents, but told our US teacher in the morning about the incident. Phone calls to parents promptly ensued.

Bora Bora

I was part of an acquisition in quaint, humble Ann Arbor. As Midwestern as fresh-pressed apple cider. The CEO of the acquiring company eventually dropped in, and we carved out an hour to hear as we squeezed into our 20-person conference room.

An HR administrator had to fly in special from California just for this visit to prepare for him. I don’t know all what she had to do. I do remember a very special component was ensuring that the CEO had Diet Pepsi Cola Wild Cherry chilled in the fridge for him in case he got thirsty. It was even labeled with his name and with orders for us not to drink it. Apparently the Diet Pepsi Cola Wild Cherry was a key to his success.

The CEO was dripping in his suit and watch that cost more to buy than I made in a year at the time (I looked it up).

I’m really tired. I’m just stopping in on my way back from Bora Bora from our top sellers event. Who here’s been to Bora Bora? Raise your hand!

One person raised theirs. Some people I could see pulling out their phones to look up Bora Bora. I wasn’t too familiar with my French Polynesian islands either to be honest.

It was a real Mitt Romney moment. Like when he made a bet for $10K in a national debate like it was nothing.

Anyways, in a 1:1 session with the former CEO of the acquired company, apparently some advice was dispensed about a difficult employee. The big shot’s advice?

It’s the a*$holes who get things done.

Ah, two keys to apparent success. That and a cold Diet Pepsi Cola Wild Cherry.

Strange Things

Fri, 03 Feb 2023 00:00:00 +0000

A collection of strange experiences I’ve had in corporate America.

Corporate Espionage

I worked at a Fortune 500 via acquisition. I had high-privilege access to the acquisition company’s systems. One day I was asked to join a video conference with a member of the security team. They informed me that the parent company’s corporate espionage team suspected insiders in the broader org.

I was given a few usernames and asked to see if they appeared at all in our systems’ audit logs. I was instructed not to search for those usernames on the corporate directory or elsewhere. They were unsure what other systems these individuals may have infiltrated, and didn’t want searches to tip them off that the mole was compromised.

Never heard where that one went. But it was my first personal exposure to the reality of corporate espionage.

Lesson learned: Spycraft persists, especially in high tech fields.

Stickup Stills

Another time I was contacted by corporate legal from a Fortune 500. They were cooperating with law enforcement on a local crime. Someone had robbed a convenience store at gunpoint wearing a jacket with a company logo on it. This was a rare jacket that not many employees would have. They suspected that it had been donated to an organization and picked up at a secondhand store. However, the police still wanted to rule out that this wasn’t an employee in our office.

I was a long-tenured employee who had interacted with many folks in that office throughout the years. Hence why I was brought in. Before they showed me the security camera stills, the attorneys briefed me that it’s natural to want to be helpful. However, unless I really did recognize this person don’t say something like “Hmmm…well maybe it kind of sort of looks like…” I should just state whether I recognize the person or not.

Lesson learned: Answer the question you’re asked, nothing more; being helpful sometimes isn’t.

Embargoed Vulnerabilities

Not too surprising now, but when I was early in my security career I learned that vulnerabilities get embargoed. This works similar to press stories.

A company may discover a bad vulnerability in a piece of open source software. They share the issue privately with the organization that maintains the software of course to determine a fix. Before the vulnerability is publicly disclosed, the discovering organization may decide to share it with critical partners under embargo.

The idea is to give them a heads-up and a chance to fix the issue on their own systems before it goes public and is explicted. The partner is sufficiently important to the main organization’s security that just letting them patch the issue after public disclosure would present sufficient risk.

I’ve worked a few of these. You’re always under pressure to remediate on a tight timeline and sharing context for the changes with as few people as possible.

Lesson learned: It’s nice to have powerful friends who find you important.

Performance Management

I was working for a company that had rolled out a new tool for managers to complete performance reviews. There was weird UI issue that kept recurring, so I decided to open Developer Tools. While inspecting the issue I noticed something weird about the network calls.

Two parameters were being passed when loading a performance review: the ID for the employee’s review and the email address of the person loading the review.

{
    "employee_id": 1234,
    "email": "erik@example.com"
}

Huh, that seems goofy. What happens if I change employee ID to my own and my email to my boss’s?

{
    "employee_id": 5678,
    "email": "john@example.com"
}

Sure enough, there was my performance review. Fun! I reported it to HR, an incident was opened, the vuln was fixed. They ended up pulling audit logs to see who else may have accessed records with a mismatch between the signed in employee and the email address.

Some time later, I mentioned that I found this issue to another employee. They told me about a case from years before where interview scorecards were being exported to the company’s data warehouse with limited access controls. Numerous employees found out and queried the data before it was reported.

Audit logs were pulled again and HR reached out to the employees who accessed the data. Most employees either selected a few rows or just their own interview results.

select * from interview_scorecards limit 5;
-- or 
select * from interviews_scorecards where username = 'erik';

But a few had queried well beyond personal curiosity it seems.

Lesson learned: If you find a vuln don’t abuse your discovery. Report it responsibly and move on with your life. The audit logs don’t lie.

Patent Programs

Mon, 31 Oct 2022 00:00:00 +0000

A VP of Engineering I used to work with asked me the following:

Would you happen to know of any patent programs or processes that work?

I was trying to give feedback on some proposal on how to make our patent program work better, and thus interested in leveraging what works for other people. Basically wondering what prior art there is.

My short reply was “no”. Every program I’ve ever participated in has been roughly the same shape.

Submit your idea in a tool
Maybe get followed up for more info and to proceed
Otherwise, you’re quietly told “thanks, better luck next time”
Even when your idea gets filed, you have no clue what parts are patentable, especially compared to other rejected ideas that seem more novel

I started thinking how to build a differentiated patent program at least two orders of magnitude more effective than the norm.

Usually the extent of education and feedback received by employees is limited to ideas like these from the Schox Group:

Don’t search to see if the idea is already filed, otherwise damages are tripled
First-to-file vs first-inventor-to-file
We’re building a defensive patent portfolio

The rudimentary nature of the education and feedback is probably done with good intentions. That way potential inventors don’t worry too much about their ideas or whether they are worthy of being submitted.

However, this is a bug, not a feature for patent programs. The typical engineer thrives off pattern matching and case studies. Here are two ideas to correct this.

First, commit to feedback after submission. This creates a positive feedback loop that leads to more and better ideas submitted.

If an idea doesn’t qualify, spend 5-10 minutes with the submitter on why (in layman’s terms). This helps them get resolution and to try again with a new perspective.

If an idea does qualify and is worth filing, spend 15 minutes educating the submitter on what aspects led to this result.

Similarly, let them know how to better frame their submitted ideas.

Second, give a quarterly case presentation on a patent the company successfully prosecuted. This improvement is less personal but magnifies the reach of the education. It could slot into an existing tech talk program.

These should be joint presentations between the inventor and an attorney from the patent team. The inventor presents their idea and its use at the company. Then the attorney discusses what was worth filing in the idea. Close with a final plug on how to submit ideas and Q&A.

These talks resurface the program to the fore of employees’ minds and help them build mental muscle in thinking about their ideas. Of course, the speakers must be engaging. If a company did these talks well, I guarantee they’d have strong attendance.

These ideas are relatively cheap to try for a year. I’m convinced they’d work at a sizable company and would be willing to place a gentleman’s bet on it. 💡

Every Flaw

Tue, 25 Oct 2022 00:00:00 +0000

Craftsmen are cursed in any endeavor they pursue. The curse is the gap between their vision and reality.

It’s not easy to live up to the imperfection. Some cultures develop entire world views dedicated to accepting the imperfection. For example, the Japanese have wabi-sabi.

Whether it’s career, how a project was delivered at work, or even the result of a home improvement project, we’re burdened with the gap.

We enjoy the finished product. Yet knowing where expectations didn’t meet execution forever taunts us.

The sense of imperfection attenuates with time. But it never dissipates.

I used to think that agonizing over the delta was my curse alone. That those who I clearly knew to be masters in their craft didn’t suffer this. I was wrong.

I only got 25% of what I wanted on Star Wars.

I can see all the scotch tape and the rubber bands that are holding it together.

But that’s how movies get made. They don’t get made right. They get made the best possible way under the circumstances.

– George Lucas, Episode 2, Light & Magic

Lest we think it’s only the famous who suffer this.

You live it and breath it for the whole time you’re doing it…

In my head I can visual the images I’ve been looking at…

I don’t think I’ll ever be satisfied no matter how well I’ve made a piece. I just think it’s in my nature to always think I could have done better.

– Ricky Pittman, Chicken Wire Sculptor, Handcrafted Hotels

Where we see perfection, they see anything but. Welcome to the club.

Square Root

Sat, 22 Oct 2022 00:00:00 +0000

Occasionally I’m asked why even seemingly simple software is riddled with bugs.

For example, a friend or family member asks why their favorite app had an obvious bug recently.

Come on, you just have to upload photos and let me see others’. Why is that so hard?

I’ve developed a small illustration to help convey the challenges. Let’s start simple and build up.

Imagine you’re asked to make a black box. This is our app.

The user punches in a number and it returns the square root of that number.

Ok. Let’s go!

The user inputs zero. And we return…zero.

The user inputs four. And we return…two.

So far so good? Great!

Actually, we already have our first bug. Maybe. Both two and negative two are square roots of four.

Did we want our black box to return only the implied (positive) square root of the input? Or did you want any of the possible square roots?

I can hear the protestations now.

But you didn’t tell me that’s what you wanted!

You don’t say!

This is the exact problem engineers face when developing software.

Our illustration had one simple job, and we’re already into the ambiguity of requirements.

Now imagine developing high-stakes software. Maybe it needs to send rockets to space. Maybe it will process your mortgage application screening you for creditworthiness. Maybe it will drive your car autonomously.

It’s a minor miracle software works as well as it does. There’s an army of engineers employed at the companies building the software we use daily.

And they care about what they build. A lot. They sweat the details to minimize issues like the ones above.

In spite of all of this, there will always be bugs.

Leader Lore

Fri, 21 Oct 2022 00:00:00 +0000

Silicon Valley lives off leader lore. Whether truth or myth, the lore forms and perpetuates an image of the leader.

Stories abound how Steve Jobs acted in a particular situation.

The designing of iPhone was not the first time that Steve Jobs put his foot down about this kind of Simplicity. A former director of product marketing at Apple, Mike Evangelist, has told the story of one of his first meetings with Steve Jobs, a meeting that took place in 2000 inside the Apple boardroom.

Mike’s team had been charged with developing a simple way to turn a home movie into a DVD, an app that would later show up as iDVD (one of the iLife apps). He and a partner worked hard to develop their ideas for an interface that would be user-friendly enough for Steve, and prepared to share their work with him by creating all kinds of sample screens and verbal explanations.

Mike was shocked when Steve Jobs walked into the room, ignored their work, and walked right up to the whiteboard. “Here’s the new application,” he said. “It’s got one window. You drag your video into the window. Then you click the button that says ‘Burn.’ That’s it. That’s what we’re going to make.”

— Ken Segall, Insanely Simple

While Steve Jobs focused on radical simplicity, Bill Gates was said to test the depth of your knowledge to determine if he could trust your recommendations. In meetings he’d relentlessly ask probing questions, a technique known as dip-sticking. Eventually he’d bottom out your knowledge and either be satisfied or not. If you won his trust, you were free to proceed and gained his esteem.

Here I contribute a few pieces of lore to the canon. The stories are second-hand. Minor details may differ from reality (e.g., it was 20 hours, not 24 hours). Regardless, the lore supports the reputation.

I interviewed a candidate who worked at Tesla. At one point the Tesla build process was taking over 24 hours to complete, delaying development. A group of engineers was selected to improve the situation. Elon dropped in on their project kick off. His ultimatum was that the build must take less than 5 minutes. He’d be checking in daily on the team’s progress. They were incredulous it could be done. They got it down to 30 minutes. Good enough. Elon moved on.

Another time I worked at a company that built internal IT software. We were working to sell Tesla. Elon didn’t like an aspect of the user interface. The sales team joined a call with Elon and our CEO. We weren’t willing to change our roadmap for a CSS customization. We moved on.

Both stories are congruent with Elon’s self-acknowledged nano-managerial style.

Speaking again of build systems, let’s talk about Sergey Brin. Sergey took a different tact from the ruthless grilling of Gates. His humor was a weapon to cajole performance and make a point. A coworker and ex-Googler relayed a story when he was in a meeting with Sergey. One team was reporting out that their build was taking over 30 hours to complete (apparently a common theme across these companies). Sergey remarked,

30 hours? Really?! I can crawl the world in 30 hours.

Tails between their legs, the team fixed their build lest Google crawl the web faster than their team’s build.

Here’s one more as we round third and head for home. Google is famous for the criteria that something must be a $1B idea to even be considered. An ex-Googler shared that Sergey was reviewing the Google Earth Enterprise growth charts with revenue sitting in the low 8-figures. He squinted at the chart and gestured with his finger drawing a line,

Yeah, if I extrapolate way out, we might hit one billion. Though, I’m not sure if I’ll retire before that happens.

Google Earth Enterprise was retired in 2015.

Local Maxima

Mon, 25 Apr 2022 00:00:00 +0000

A regular topic occurs in professional coffee chats. A leader has inherited a particularly dysfunctional $X: team, project, product. It’s the type of mess that if you knew what you were getting yourself into ahead of time, you never would have accepted the challenge.

The question the person wants to answer is, “Where do I start? How do I know I’m working on the most important thing to fix in these situations?”

The good thing is that identifying the most important thing doesn’t actually matter. And that’s for two reasons.

First, you’ve likely inherited a complex system of interrelated problems contributing to dysfunction. The likelihood that your mess is caused by just one thing is very low. Otherwise, the previous leadership would have fixed it by now. In reality, there are a dozen contributing factors to your mess.

When the person frets that they’re not working on the most important thing, they’re concerned about the search space. They want to make sure that they locate and fix the global maxima of their problems. This is a strong intuition, but slightly misguided. What they’re trying to avoid is fixing window dressing. For example, if they’re reorging an underperforming team, focusing on a team logo is not that important.

The reality is that there are a multitude of high impact things that they could work on. Each of those items has a local maxima that really isn’t that far off from the global maxima. Instead of focusing on the highest leverage item to fix, just find a high leverage item and fix that.

Second, progress you make on material items creates a positive feedback loop. Pounding down a high leverage local optima reduces the total volume of outstanding problems. The team is freed up to focus on fewer things, the system is made less complex, momentum builds. Goto new problem.

This is not dissimilar to the idea behind the “debt snowball” in personal finance. Tackling your biggest, hardest debt may not always be the right thing. Completing something tangible allows the team to accrue wins and carry that forward into the next problem.

There are exceptions to this rule though; issues you absolutely must fix. I was once part of a reorg. It was clear that hiring was broken, and we weren’t selecting for the people we needed to turn things around. That was a must-fix issue. From there though, the wins (slowly) compounded.

Impostor Syndrome is Fractal

Sun, 24 Apr 2022 00:00:00 +0000

Comparison is the thief of joy.

– Unknown

For those of us who suffer from impostor syndrome, and it’s not everyone, the mark it leaves is indelible. The constant comparison and concern about delivering and keeping up both fuels and depletes. Strivings completed provide an all too brief reprieve. They are ended soon by the question “What’s next?” as you watch your contemporaries push forward while you rest.

Impostor syndrome is fractal. It doesn’t fade away. As cyclist Greg LeMond said, “It never gets easier, you just go faster.”

No matter your position or talent, the deficit between “their” skill and yours feels constant, infinitely nuanced, and ultimately un-closable.

Say you’re a beginner whose skill is 10. Examining someone a bit ahead of you at 100, the delta seems so wide.

But even for an expert–say a world class operator in their field–who executes at 1000, the delta between them and someone at 1010 is perceived to be just as wide.

This is demonstrated in episode one of Get Back by Peter Jackson.

George Harrison: …the difference between me and, say, Eric, I’m just another guitar, sometimes playing bits and sometimes singing…So…I feel now I can play things. I can learn things that will sound okay, especially fast fingering like that.
Paul McCartney: It’s jazz, man.
George Harrison: Not really, Just Eric. He’s very good at that. At like, improvising and keeping it going…which I’m not good at.

The words alone don’t justify the exchange. George suffers throughout the clip. Here’s one of the most famous, most accomplished musicians forlorn over the gap between himself and Eric Clapton. Mind you, this is after Beatlemania, after fame and fortune has been attained. And yet George Harrison feels under-accomplished, trapped, and lacking.

Let’s take a more modern example from the podcast episode How I Built Resilience: M. Night Shyamalan

Guy Raz: How did you overcome self-doubt?
M. Night Shyamalan: You don’t. You don’t. I have it right now in front of you. I mean, I just came from the editing room and I have all these issues with this reel and I’m trying to figure it out and the voice inside of you says, “You’re not, you’re not going to figure it out.”…I’m scared to write [my next movie]. Maybe today I won’t be able to think of that thing. My oldest daughter is an artist and I said, “That is your plight. You are going to wrestle with your demons every single day of your life.”

If you suffer from impostor syndrome–and I think “suffer” is an appropriate verb to describe its effects–you’re in good company.

Essays by Erik

Where to Put the Bolt

The (AI) Road Ahead

Crushing the Competition

Computers as Jobs

Winter Is Coming

Micro-pessimism

Macro-optimism

The Weight of Privacy

What Is Privacy?

Why Care about Privacy?

Privacy Engineering (and How It Is Different from Security Engineering)

Selected Case Studies and Lessons

Was clear, plain-terms consent obtained before data collection or use?

Inversion

Did the reason for collecting the data remain consistent?

Inversion

Is access to data limited to proper use cases?

Inversion

A Vector, Not a Destination

The RSU Cookie Jar

Double-Trigger RSUs

Susan’s RSUs

The RSU Cookie Jar

Stripe’s Fundraise

22% vs 37%

Of ISOs and Edge Cases

Stock Option Background

Alternative Minimum Tax

Deal Close

83(b) Elections

Qualified Small Business Stock

The Sacred Timeline

More Strange Things

You’re a Hard Man to Find

Mexico Bus Robbery

Bora Bora

Strange Things

Corporate Espionage

Stickup Stills

Embargoed Vulnerabilities

Performance Management

Patent Programs

Every Flaw

Square Root

Leader Lore

Local Maxima

Impostor Syndrome is Fractal