Millions more workers could be indirectly affected by changes to the salary sacrifice scheme announced in the November budget, a former pensions minister has warned.