Week 4: Brainstorming MVPs

Key Learning Moments

  1. The DoS, and specifically analysts beneficiaries, would find it useful to have an internal wikipedia-like page compiling information related to sanctioned entities;
  2. The permeability of sanctions enforcement is not necessarily a negative thing: it can be a net positive for US strategic interests;
  3. US blacklisting practices are stricter than UN ones.

Experiment Results

This week, we tested three hypotheses:

  1. Different governmental agencies conducting enforcement do not adequately share information related to their activity in this respect;
  2. Although a list of bad actors already exists, and it is managed by the US Treasury, the list it not comprehensive and under-inclusive: a less formal blacklist would facilitate impacting shippers involved in violations;
  3. Not all sanctions are created equal: some have a stronger impact than others. The government can and should prioritize their enforcement strategically.

Our interviewees this week involved people from State Department and Intelligence, as well as a wide range of experts in North Korea economy and markets. We also had the opportunity to interview a plurality of journalists and bloggers.

As to the first hypothesis, it was difficult for us to fully prove or disprove the lack of information sharing within different agencies. The issue is classified, and as such we were unable to get a clear-cut answer. We were able to find out that a task – force agency does exist for the purpose of information sharing. However, the actors that are not part of it do not benefit from this information sharing – for this reason, they should be the focus of our further research.

Secondly, we tested the possibility of creating a less formal lists of “bad actors” and its potential utility. Notably, there was no consensus between intelligence and other enforcement authorities, to whom we submitted the idea. The primary concern with the list is the costs it would bear: as such, further pursuit of this idea will require an attentive balance of its costs and its benefits.

Lastly, we discussed with our interviewees the differential impact of sanctions. We learned that, in some cases, sanction violation can be a positive outcome for the US strategy. Whether and how the US could better prioritize enforcement remains unclear, but we plan on engaging our sponsor and, potentially, other intelligence contacts to further elaborate a possible prioritization scenario.

Our Potential MVPs

Based on these findings, we started drafting three possible product directions.

First, to encourage better sharing of interagency information, we could develop an internal Government Sanctions Wikipedia. Having a centralized database, in fact, would significantly decrease the time necessary for State Department negotiators/desk officers to perform démarches, once intel is received regarding a sanction-evasion related activity. We know a similar, external tool, called Pyongyang Papers, already exists and is open sourced. The information is gathered from media outlets, or other public reports. However, we believe the database is flawed. We plan on digging in more deeply in the upcoming weeks to flesh out how it can be improved and effectively deployed.

Second, to increase barriers to ship-to-ship transfers for North Korean ships, we could build a government sponsored, yet informal “Watchlist” of “bad actors”. This product would achieve two goals: first, it would name and shame the companies that help with sanction evasion, leveraging the possibility of a deterrent effect; second, it would prevent shipping companies from claiming “ignorance” when conducting transfers to North Korea.

One idea we had is to rely on technology, such as automation, to help deploy a more effective product than the currently existing Treasury OFAC List. In fact, as it came up in one of our interviews:

“…if  there were a way that was more dynamic in helping to understand who you are dealing with – that would give you a better shot at doing enforcement. Being able not to be manually update these tools could be extremely valuable. There might be ways to look at previous cases where we know that there was a network going on and train through AI a system to look for similar patterns.

Third, we will continue brainstorming on a potential prioritization tool. Before further focusing on it, we will discuss with our sponsor whether this direction might reveal itself to be useful or not.


This week, we focused on conversations with a wide range of North Korean experts, including former State Department and Intelligence officials, journalists and policy experts.

  1. William Brown, Former ODNI
  2. Alastair Gale, Wall Street Journal
  3. David Slayton, US Navy, Hoover
  4. Andy Kim, Former CIA
  5. Chloe Chung, Korean Foreign Ministry
  6. Todd Buchwald, Former Ambassador for Global Criminal Justice
  7. Richard Johnson, Nuclear Threat Initiative
  8. Kyle Ferrier, Korea Economic Institute of America
  9. Andray Abrahamian, Chosen Exchange
  10. Eddie Fishman, Former State
  11. Joshua Stanton, One Free Korea
  12. Sig Hecker, Stanford Nuclear Energy Expert
  13. Matt Prusak, Korean Institute
  14. Michael Matheson, Former State

Week 4 Customer Discovery

To prepare for week 4 we are hoping to dig up a few more pain points or gain areas that might help us solidify our product-market fit on our mission model canvas. We plan to accomplish this by hitting one last round of North Korean experts, focusing some on economic and market dynamics. We also want to confirm that certain assumptions about who benefits from smuggled petroleum are correct.

We also want to test an idea that came out of an interview regarding the narrowness of the OFAC SDN list and the possibility of providing gains to corporate executives concerned with legal and public relations exposure while also supporting the US lead negotiator by weakening North Korean access to smuggled petroleum products.

Weekly Hypotheses:

  1. Refined Petroleum imported illegally through ship-to-ship transfers is primarily benefiting the North Korean state and not quasi-private black market actors.
  2. The Treasury Department OFAC SDN list is significantly underinclusive of all relevant bad actors and publishing a wider ranging list would reduce North Korea’s access to sanctions evading markets.


We will conduct Customer Discovery interviews to test our hypotheses. We especially hope to pull out of each interview one or two ideas for areas that need attention and possible future MVPs.

Week 3: Moving away from at-sea interdictions

Key Learning Moments

  • We learned that there are very few actual at-sea interdictions performed to enforce refined petroleum import sanctions.
  • We learned that reforming the flag state approval process is unlikely to meaningfully improve refined petroleum interdiction capabilities related to North Korea.
  • There formal US Treasury Sanctions list likely does not include all publicly known bad actors and thus is underinclusive of potential targets.

Experiment Results

An Underlying Assumption for our active hypotheses was disproved.

We had assumed without proving that a significant number of at-sea interdictions were occurring or were predicted to occur related to efforts to curb North Korean refined petroleum imports. We were quickly disabused of this notion across several interviews. Despite some serious at-sea interdiction efforts related to missile and nuclear technologies, at-sea interception of refined petroleum smugglers is incredibly rare (and even when it does occur, it rarely results in a boarding). We also learned that what interdiction do occur are far more likely to be driven by active intelligence rather than a patrolling vessel happening upon an illicit scene.

Hypothesis 2, related to shortening the flag state approval process, was also disproved.

Our vision for eliminating a pain point around flag state approval procedures was undercut by the discovery that at-sea interdictions are rare and are often directed in advance based on active intelligence. Because the model we were working with involved a ship commander having to wait for a convoluted flag state approval process to occur, we thought it was an area for improvement. Now that we realize interdiction missions will usually be planned in advance, there is little likelihood of flag state approval holding up the process.

Mission Model Canvas Impacts

We need to work to better identify our final beneficiaries. We will have a difficult time solidifying any plans if the only person we know we want to benefit is the lead US negotiator. This problem is intertwined for us with identifying what aspect of the problem we want to influence given the wide variety of actors involved with the greater North Korean sanctions effort.

Moving away from at-sea interdictions as our focus changes the networks we expect to be working within to effectuate change. Instead of ship commanders and tactical leads, we may find corporate shipping executives and port authority managers to be more relevant. Similar to our difficulties with identifying beneficiaries without having first solidified which “market” we hope to impact, it is hard to identify future partners or opponents of changes we might make.


We learned early on that focusing on at-sea interdictions was unlikely to be fruitful so we tried to engage a wider network of informed experts to help identify new areas of influence to investigate.

  1. Adam Smith, Former Senior Advisor OFAC
  2. Andrew Moore, Former State Dept
  3. David Debartolo, Former State Dept
  4. Katie Visser, Special Counsel to the General Counsel, Dept. of Defense (on background)
  5. Jeremy Weinstein, Former USUN, Former NSC
  6. Van Jackson, Professor at Victoria University in Wellington, New Zealand
  7. Youngjoo Cho, Aviation and Korean Law Expert, Stanford
  8. Paweł Jabłoński, Department of Strategic Project Coordination, PM Office, Poland
  9. Michał Lewandowski,  OLAF (European Anti-Fraud Office)
  10. Vincent Kor, Maersk International

Week 2.5: Explaining Ourselves

One of the major differences in attacking this problem through the lean launchpad methodology vice traditional hierarchical policy organizations is that we are constantly engaging with a revolving cast of potential customers and beneficiaries instead of having a single stakeholder directing us. This gives us flexibility and access to a more diverse set of viewpoints, but it also means that we can’t take even the most basic assumptions for granted in our discussions. This has led us to realize that it might benefit us to make some of our thought processes more public in order to receive more focused feedback. To that end, we wanted to start this week’s blog post off by revisiting some of our Week 0 and Week 1 work that is directing a lot of our current thinking.

One of the central difficulties to our problem is deciding at what level of generality we can add value to an effort that is already the subject of significant and diverse attention across several agencies and departments. In order to attack this problem, we began by trying to map out the universe of actors in the following graphic:

But even this graphical tool had several assumptions and decision baked into it, so instead of beginning here, we decided to start from scratch and explain our thought processes more comprehensively. Our sponsor, the Office of the Secretary of Defense put forward our problem as one of maritime sanctions enforcement:

“Develop an approach for the Office of the Secretary of Defense to enforce United Nations Security Council international maritime resolutions in order to better compel an end to North Korea’s nuclear weapons program.”

But even this formulation makes some assumptions that we would like to identify, either for further validation or to point out what assumptions we are working from. To accomplish this, we begin with the fundamental block of this class: US National Security Interests. Our goal with this project is to develop a product that will in some small way further US National Security. More specifically, in this case, we are trying to further US National Security by reducing North Korea’s nuclear and ballistic missile capabilities.

Even this basic inference is not without detractors, as some in academia and out have argued that long term national security interests might be better served by wider spread of nuclear weapon capability. Whatever the current state of political science regarding this, we think it safe to accept that reductions in North Korean nuclear and missile capabilities will serve US national security interest based on widespread public concern, political agreement, and policy direction. Our project then, must in some way affect North Korean nuclear and ballistic missile capabilities.

There are two ways that US actors can try to influence North Korean capabilities. First, the US can try to undercut North Korea’s technical ability to develop, maintain, launch, or successfully detonate nuclear weapons and ballistic missiles. This has been an area of intense attention from the US and international community through four main avenues:

The most severe method for addressing threats from North Korea involve direct military action. For obvious reasons, this is not a favored strategy. Perhaps more directly for our work, the resorting to force involves political-strategic calculations that we are unlikely to influence through this project.

A less drastic form of direct action against North Korean capabilities involves “left of launch” methods. Though inherently appealing as a way of undercutting the threat without much cost, covert action in this way is potentially dangerous because of the internal effects if may have on the North Korean regime.. It is hard to trust that left of launch methods will be completely effective because of the complex and increasingly domestic nature of the North Korean program. Regardless of the ultimate appropriateness of a left of launch strategy, we are unlikely to add value to the decisions in that area because they are inevitably intertwined with highly classified sources and methods capabilities that we do not have access to.

A third avenue of addressing North Korea’s ability to harm the US and US allies involves missile defense. There is a long history of debates around the wisdom of deploying missile defense systems to the region. This debate inevitably intersects with great power dynamics involving Russia and China whose own missile capabilities are also affected by any defenses in the region. We also concluded that our ability to add value to US decision making or technology development in this area is low, given the immense technical and strategic complexity of missile defense systems like THAAD and AEGIS and the difficult strategic calculations that must be made, relying on information we are unlikely to have full access to.

The final major way to undercut North Korea’s technical ability to threaten US interests is by restricting its access to resources necessary to its missile and nuclear programs. This has been a major focus of the US and international efforts to curb the North Korean threat, as can be seen in some of the early UN Security Council Resolutions addressing the modern issue, which focused largely on the importation of missile and nuclear technologies. This has been an area of active efforts by the US and allies in the past and continues to be an important tool in addressing the North Korean threat. Our team, however, has tentatively determined that it will not be a major are of investigation for us for three reasons (though we are still validating this decision). First, several of our interviews indicated that there is a lot of internal focus on the enforcement of weapons related sanctions in the intelligence, military, and diplomatic communities and thus we are likely to end up duplicating internal efforts. Second, as we have learned though countless awkward silences and long pauses, these efforts are often highly classified and intertwined with sources and methods; so we are unlikely to have access to enough of the picture to determine where the pain points and inefficiencies are in the current system. Finally, there is some reason to believe the arms-embargo side of North Korean sanctions will become less effective as the North Korean program advances. Successful Nuclear and Ballistic missile testing indicates an increase in domestic competency that is less susceptible to external embargoes. There are still serious questions about their miniaturization efforts that remain untested and thus may still require outside components, but the overall progress of the North Korean program is rapidly diminishing the likelihood that technical component importation will remain a significant obstacle for North Korean capabilities.

This lead our team to investigate the other prong of the equation, how to change the calculations of North Korean decision makers on this issue.

There is a central problem to any effort to change North Korean decision making related to their nuclear and ballistic missile program: They view it as an essential security guarantee against existential threats. Thus, any effort to change their decision calculus must create enough pressure to outweigh an interest as strong as regime survival itself. There are two main ways this might be accomplished: first, by creating a separate threat to regime security that is more immediate than those North Korea seeks to deter through nuclear weapons or second, by decreasing the threat perception of North Korean leadership and offering positive reinforcement to incentivize deescalatory decisions.

This is not an easy task, and the US has few tools at its disposal to change North Korean thinking because of decades of estrangement and the counterbalancing strategic interests of China and Russia. The main mechanism available to the US is sanctions, both through the UN Security Council and unilaterally. There are serious questions about whether sanctions of this sort will ever be effective against North Korea. Despite these serious questions, sanctions represent basically the only public mechanism to influence decision makers short of force and there is evidence they have some significant effect. Most importantly, they are an important ingredient of any overall strategy to change behavior, even if they are unlikely to prove decisive on their own.

The other main mechanisms by which the US might influence North Korean decision making involve limited uses of force to change their threat calculations or covert action. Both of these provide less appealing area of investigation for our team because of limited access to classified information and concerns about ultimate effectiveness. Limited uses of force are practically precluded by North Korea’s immense deterrent capabilities against South Korea. Covert action to influence regimes has a long and sometimes sordid history, but even if it were to be pursued, we would have no access to any of the necessary information to make informed decisions about its use.

Because of this, we have tentatively concluded that sanctions enforcement is a worthwhile way to address the North Korean threat. Even if sanctions are not fully effective themselves, they are a useful supplementary tool and UN Security Council sanctions have the added benefit of engaging the wider international community in a way that does reputational harm not only to North Korean actors but other countries and private actors that might otherwise deal with them. Because of these potential benefits and our relative weaknesses in addressing other areas, we think we can safely focus on effectuating decision-maker influencing sanctions as the focus of our project.

This leads us to look back at the ways in which North Korean decision makers can be incentivized to change their behavior, either through threatening regime security or by reducing threat perceptions and rewarding positive steps. We established seven main methods(though we have not yet determined this list to be exhaustive) to accomplish this goal and grouped them by whether they involve undercutting regime security or deescalation and positive reinforcement.

Decision making around positive reinforcement, whether economic or diplomatic is centered in the highest echelons of US political and diplomatic decision-making. Like covert action, these levers will be difficult for us to use and sanctions enforcement is unlikely to weigh heavily in these areas.

Sanctions do affect the remaining levers of influence, by hurting the North Korean body politic, undermining North Korean military readiness, and restrict the leadership’s ability to win favor with elites. We have tentatively rejected looking into sanctions on luxury goods targeting elites because of concerns about effectiveness in a regime that maintains tight control over even the elite and because of concerns about the efficiency of targeting smuggling of smaller and more easily concealed goods.

Instead, we have focused on sanctions enforcement that directly undermines the North Korea regime’s security either by undermining its conventional military capability or by creating socio-political pressure via the body politic.

We are still exploring the different ways by which we can make a difference through these two ideas, but we can make a few threshold conclusions. First, any restrictions on food imports through sanctions has been widely rejected by the international community on moral grounds. There is some debate, however, as many of the current sanctions restricting North Korean access to dollars and oil might have just as significant of an impact on access to food. Regardless, we have no interest in pursuing purposeful starvation as a method of influencing the regime.

We can also tentatively eliminate looking into restrictions on access to conventional military technology and information warfare as useful areas for further research for similar reasons to our decision not to look into arms control sanctions and covert action above. We are likely to run into similar issues around classification.

This leaves two major mechanisms for us to explore further, both heavily involving UN and US sanctions and their enforcement: restricting North Korean revenue and access to foreign currency and restricting North Korean access to refined petroleum products.

The ability for North Korea to raise revenues and to generate foreign currencies is vital to the regime and has been an ongoing issue of international attention. We are continuing to investigate some of the possible mechanisms to effectuate change here. One key aspect of the problem is trying to disentangle North Korean State actors from quasi-private entities whose interests may diverge from the regime.


It has been readily apparent from the beginning that access to refined petroleum is a key pressure point on North Korea leadership, both because of a need to fuel their military efforts and because of a growing domestic and commercial appetite for taxis, generators, farm equipment, and other diesel and gasoline needing machines. The UN Security Council has thus focused much of its recent import restrictions on refined petroleum and crude oil that North Korea has the capability to refine itself. These sanctions have not proven completely effective, as recent UN reporting has made public, partially because of significant North Korean evasion through at-sea tanker transfers.

This has resulted in and increased focus on addressing the ship-to-ship transfers that North Korea is using to evade the refined petroleum cap. We have began to focus tentatively on this as our primary goal, though there are still some questions we would like to answer about the ultimate utility of spending resources on enforcement of this particular form of sanctions evasion.

There are several vectors that we are investigating to investigate where we might be able enhance US enforcement capabilities around these ship-to-ship transfers. Our continues investigation of this will be detailed more in following blog posts, but we wanted to take the opportunity here to show how we got here, what other areas we are still considering, and what our reasoning was at each step of the process.

Week 2: What fuels DPRK?

What channels does North Korea use to bypass sanctions?

No cars on North Korea’s main highway Ships coming in and out the Wonsan port

  • Ensuring that the sponsor-provided problem is a key component in the wider issue of enforcing sanctions
  • (Dis)proving our ideas as to who are the actors relevant to the problem
  • Forming a hypothesis as to what form should our solution take

This week’s goals

Overall, while in the previous week our focus was largely on who we should serve, this week we were trying to get closer to establishing what they need.

Working hypothesis:

  • Illegal maritime trade in hydrocarbons is crucial to the DPRK sanctioned weapons programme

The hypothesis has been proven

“Devoiding North Korea of illegally smuggled petroleum would metaphorically and literally deprive the DPRK war machine of fuel.”

EU Office of the High Representative for Foreign Affairs and Security

“North Korea has a $3.4 billion dollar total import economy. When we stop a $1 million dollar refined petroleum shipment, that has a major impact.”

North Korea Policy Expert

The two interviews have strongly indicated that illegal maritime imports of refined petroleum are a key part of the problem. It is furthermore the only one that would seem to be under at least partial control of US and allied assets, as opposed to DPRK’s other suspected revenue sources.

  • The key obstacle towards efficient interdictions is the diplomatic/legal procedure involved (taking upwards of a day

The hypothesis has not yet been proven despite strong indications

The interviews have provided us with a thorough overview of the marine interdiction process and the legal/diplomatic aspects behind it. In further weeks, we will try to look for possible improvements in the procedure and look for potential time gains which would allow US/allied assets to increase their interdiction efficiency.

Post-interview summary

The team has conducted 11 interviews with relevant actors and contacted another 36. The noteworthy conversations were:

  1. Allen Weiner (Professor, International law, Former State Dept Legal Office)
  2. Tess Bridgeman (Professor, Sanctions Expert, Former State Dept)
  3. Andrzej Sikorski (Current Acting Polish Ambassador to DPRK)
  4. Matthew Kaseman (OSD Korea Policy Team)
  5. Richard Nephew (Former State Dept)
  6. Cliff Johnson (Former State Dept Legal Office)
  7. Newell Highsmith (Former State Dept Legal Office)
  8. Greg Terryn (Center for Arms Control and Non-Proliferation)
  9. Jennifer Chalmers (Department of State)
  10. Matthew Kaseman (Office of the Secretary of Defense)
  11. Official (preferred not to be identified), (EU Office of the High Representative for Foreign Affairs and Security)

Key Learnings

  • Interdiction of illicit importation of refined petroleum likely has significant impact on North Korean leadership decision making.
  • Maritime interdiction often requires extensive diplomatic clearance, even when pursuant to a Chapter 7 UNSCR.

Next steps

Evaluate Refined Petroleum Maritime Interdiction Effectiveness:

  • Continue to engage experts on how sanctions are being felt within the DPRK
  • Continue to build interview chain to State Dept Negotiators/White House actors

Improving Flag State Approvals:

  • Seek out ship commanders, State Department Desk officers, Flag State Post personnel, and common Flag state diplomats/experts to better understand the clearance process
  • Identify where standing Government groups have had success in similar areas
  • Formulate potential international agreement that might eliminate need for case-by-case Flag State approvals