What is the unit of the generalized cost function? How can I determine the coefficients correctly?

There is no implicit unit for the generalized cost, so you can freely choose it. You only need to consider that travel time is measured in seconds, distance is measured in meters and that the link cost has no implicit unit. So you need to choose the coefficients both as a weight and conversion factor to convert from one unit to your chosen unit.

Example: You choose $ as cost unit.
Now you need to determine the worth of one second of travel time in $ and also the worth of 1 meter in $ (if you want to include the distance in your calculation).
If you use any link cost, you would define these as cost in $.