This question can be answered rhetorically in a number of ways without going into the pragmatic answer of where they should be at the start of any frame of Snooker or game of Billiards.
There is firstly the question of where they should be between matches when a table is being prepared for the next one. The answer to this is simply ‘in their box’. To leave the balls in the pockets or place them there whilst the table is being brushed is not an acceptable practice. Any dust raised and brushed into a pocket containing the balls will result in those balls being covered in chalk and dust and then in need of cleaning. What a waste of time and effort!
Another interpretation of the question is where should the red balls be during the course of a frame after being potted or pocketed? Well, this largely depends on the table and what type of pockets it has. If it is a standard table with Empire rails then those rails will hold 5 balls before becoming a little crowded making it difficult to remove any colours in need of re-spotting. As there are a total of 15 reds and there are 6 pockets it is a simple sum to see that no more than 3 red balls need to be in any one pocket at any time and the movement of balls in the top pockets to the rails at the bottom and centre of the table is something that should be accomplished almost automatically by any good referee and this can be done one hundred per cent unobtrusively. If there are reds or a red in the top rails they can be retrieved whenever a colour has been potted or pocketed in that pocket and in need of re-spotting. It is a good idea to do this even at the start of a frame with just one red in either of the top pockets in anticipation of later in the frame when they could be overcrowded without an opportunity to approach them, as when no colours had later been potted into that pocket. When an opportune moment arises, they can then be placed in the centre or bottom pocket rails. Retrieval of balls can also be done in the time between a player leaving the table and before, or as the incoming striker is approaching. Balls should be held in the hand until the opportunity arises to place them in a selected pocket, certainly not put into a jacket pocket containing bits of fluff and other detritus as has been observed on one occasion by a referee.
Empire rails are also available as a ‘De Luxe’ set which have two extra-long ones in the set of six for attachment to the top pockets, these will hold 7 balls before becoming too crowded and will usually result in not having to move any red balls during a frame at all except in rare circumstances. These rails however are not conducive to efficiency by the referee during a game of Billiards, with extra movement or a much longer reach being required to remove balls from them to return to play. Table 5 at Yarraville has these rails which became so annoying during a Billiards match that the table has since had cloths inserted during Billiards tournaments to block the exits of the top pockets, making for easier ball retrieval.
There is also an antique table in Government House that should really be used only for Billiards. This table has pockets similar to those with Empire rails but instead of those rails which will hold up to 7 balls it has a system which will catch only one ball. They are pockets open at the bottom as with Empire rails but with a catcher made of brass in the shape of a shallow cup that allows the referee to easily access any ball dropping into it. The downside is that only one ball can sit in this ‘cup’ contraption at any one time, almost perfect for Billiards referees but a nightmare when officiating Snooker matches.
If a table does not have Empire rails but closed expandable pocket nets as do the tables at the R.A.C.V. City Club, then those pockets are capable of holding many more balls than seems possible and it will only be necessary to empty them on very rare occasions. When these pockets only have one or two balls in them and a colour needs to be extracted during play it is advisable to lift the bottom of the pocket from underneath and retrieve the colour with the other hand. This way a referee can be sure of not losing shirt buttons or cuff links or jacket sleeve buttons and also of picking the correct ball to extract.
The only other consideration of where the balls should be is when they are in transit or being stored. Room temperature is the optimum state and they should be kept in a cool place in summer and protected from extreme cold in the winter.