Its hard to do those things because you don't know who is really serious about making changes because you're still chasing an end result. The end result is the final determination of their efforts. This is how we got where we are. We throw money at them because it's unfair to cherry-pick. We try to claim success, but those people would have changed anyway.
As far as asking employees of the system what works, you have to look at who is self serving to make a name for their selves vs the completely jaded. Too much human variable.
They say the pendulum swings from treatment to punishment. Would you believe me if I told you that research has found no change in recidivism regardless of where we are at on the swing? 31 to 33% will be back in prison within 3 years of release....regardless.
That's true. It doesn't work. I like your original idea of special pills with instant results.