The purpose of the generator light is to let you know when the battery is being discharged. Of course that is about all it tells you, an ammeter is a better deal and a voltmeter is better yet. The symptoms you are describing about the flickering is the result of normal regulator operation as the voltage/current regulating relays inside the regulator cycle on and off in response to the varying load conditions. If the generator stops charging entirely, the indicator light will shine brightly like it does when you turn the key on before starting the engine. During daylight operation with no lights on the system voltage is high enough to keep the light turned off, but at night with the lights on there is enough voltage drop in the system to allow the light to glow faintly, plus the fact that in the dark it is easier to see the light.The light goes off when the voltages across the system are balanced, and whenever there is a voltage imbalance the light will glow in proportion the amount of imbalance. Part of this is caused by the fact that you have a 40 year old car with old, tired wiring and connections that are undoubtedly corroded to some extent. I still run the stock generator/indicator light in my 68 VW Bug with good success, I would suggest that you invest in a good voltmeter and then you can make a much better informed decision about whether or not to upgrade to an alternator. There are some very good reasons why the car makers went to alternators, but if you don't place excessive demands on your generator it will still get the job done.
Joe