Recursive self-improvement is a process that gets better at its own job each time it runs.
TL;DR
It sounds like science fiction and mostly is not. A validation process that reports which of its own rules it cannot check is already improving itself — it just does it slowly and in writing.
The prerequisite is measurement. Nothing improves recursively if you cannot tell whether the last change helped.