Your engine size has little to do with main jet size. A Holley has four fuel circuits - Idle (adjustable with the mixture screws), Main ( metered with jets), Power (extra fuel for high power and WOT, activated by the power valve) and acceleration (pump enrichment).
BUT.....Before you mess with anything in the carb, you have to do three things. First, make sure the ignition is good. Second, Check the ignition. Finally, after you're sure the ignition is good, check the ignition system.
You get the point; a lot of things that may seem like carb problems are simply the result of not lighting the fire properly. An off idle bog could be caused by improper timing, for example. A "lean miss" could be a bad point gap.
IF....your ignition is good, the first thing to check is the accel pump circuit. You should be getting a good shot of fuel as soon as the throttle begins to open. If not, check the diaphragm and the check ball.
If the bog is not off idle, but rather at WOT, you need to check the power valve circuit. Make sure the power valve is working and that the passages are clean.
Main jets are what the engine cruises on when you are not idling, accelerating, or at full throttle. Out of the box, the carb is probably a little rich, but that won't have anything to do with engine size. The jets are sized to deliver a ratio of fuel to air. If the main jets are sized to deliver a 12:1 Air/fuel ratio on a 5.7 liter V8, they'll deliver a 12:1 AF ratio on a 3.3 liter six. For every pound of air that enters the carb, the main jets will deliver a proportion of fuel, regardless of engine size.
You have a little more work to do before tearing into the carburetor, I think.
