Reminder: Prompt Injection Vulnerability Still Unresolved
Software developers building on top of ChatGPT/GPT-3/4/LLMs need to be aware of the prompt injection vulnerability, as there is still no known robust fix. A 100% reliable fix is needed to prevent malicious actors from exploiting this security vulnerability.
Simon Willison
Creator @datasetteproj, co-creator Django. PSF board. @nichemuseums. Hangs out with @natbat + @cleopaws. He/Him. Mastodon: https://t.co/t0Mrmo0Z2K
-
Your regular reminder that prompt injection still does not have a known robust fix, so if you're building any software on top of ChatGPT/GPT-3/4/LLMs you need to assume you are vulnerable to the attack, and that things like your "system prompt" are effectively public information https://t.co/qz9gvzcvcx
— Simon Willison (@simonw) April 11, 2023 -
More notes here: https://t.co/tZxhwFNrvO
— Simon Willison (@simonw) April 11, 2023 -
If anyone has seen a 100% reliable fix for prompt injection I (and many others) would very much like to hear about it!
— Simon Willison (@simonw) April 11, 2023
99% effective fixes don't count, because this is a security vulnerability, so that 1% WILL be found and exploited by adversarial attackers -
Here's a bunch of stuff I've written about prompt injection (16 posts so far) https://t.co/t1wYGJU8BA
— Simon Willison (@simonw) April 11, 2023 -
So maybe think carefully about prompt injection before giving your LLM system the ability to make API calls or access private data... especially if you combine both of those and don't want someone injecting a prompt that exfiltrates your private stuff!
— Simon Willison (@simonw) April 11, 2023 -
This is interesting: on ChatML, @OpenAI state "This gives an opportunity to mitigate and eventually solve injections" - so there's a lot riding on what they mean by "eventually" there!https://t.co/9JhaJapxMS
— Simon Willison (@simonw) April 11, 2023