It sounds like your drive wheel rollers are much to loose on the pins.
The outside diameter of both the drive and idler wheels were originally 18 1/4″
The PCD of the drive wheel rollers were 17 1/4″
The drive wheel rollers 15/16″ diameter with a bore of 1/2″
The pins were 7/16″ diameter. (I would use 11mm stock)
The rollers were hard (probably case hard) and the pins were soft but from a tough steel.
Neil