Are lightweight wheels actually worth it?

Yes, because wheel weight is both unsprung and rotational, so shedding it helps more than the same weight elsewhere. Lighter wheels let the suspension follow the road better for more grip and a calmer ride, and they reduce the mass the brakes and engine must fight, sharpening both. That's why flow-formed and forged motorsport wheels command a premium. Just don't chase weight so hard that you drop below a wheel's load rating for your car.