We challenge these constraints by demonstrating that token-level iterative optimization can succeed without gradients or priors. We introduce RAILS (RAndom Iterative Local Search), a framework that operates solely on model logits. … Crucially, by eliminating gradient dependency, RAILS enables cross-tokenizer ensemble attacks. This allows for the discovery of shared adversarial patterns that generalize across disjoint vocabularies, significantly enhancing transferability to closed-source systems.
Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks (Promptfoo catalog entry here).
i think it is cool that this opens up more cpu based adversarial suffix generation. i started testing a bit with small models, it doesn’t feel as transferable as the paper claims. but atleast feasible to test these types of generation without a GPU.
also related GCG.