Defense only factors into the ‘main’ hit of an attack. Some abilities are affected by this, others aren’t… and it would take a while to list them all. But, I can say that anything that doesn’t use the attack stat isn’t going to be affected by defense (including the extra speed damage calculation). Cleave, Splash, and quite a few skills do use the final damage amount in their calculation. Usually if it says ‘extra damage’, then no defense is taken into consideration. That would include the 150% luck skill, the 50% speed skill, and Calamity. Then you have weird ones like the Mortarman. That one calculates the damage as you would have hit the creature, then distributes it to all the creatures kind of like splash would, but all enemies just take that splash damage.
The best thing to do is just to get familiar with the different skills, because a lot of them have fairly unique rules.