Will my ex get custody at paternity court if he hasn't been in the babies lives or helped with him at all?

He has a right to visitation, but most likely, not custody. To avoid drama, meet at the police station to dropoff and pick -up the children during visitation exchanges. Have someone with you, ready to record the interaction. 

Avoid stressful situations and have AS LITTLE contact with him as possible. That loser has shown his true colors.While in court, ask for a child support order.So, you have a self absorbed baby daddy you'll have to deal with for the rest of your life. Set the ground rules now and DO NOT allow him to manipulate you, Ever! Your contact with him should be kept at a bare minimum. Remember, heis the baby daddy but heis also the enemy! Nun let your guard down.

If you can prove you have been the only one caring for the babies, financially supporting them then no he most likely won't get custody. Maybe visits here and there. Make sure you bring if you have them any messages from him showing his real self. Or anything if you ask him are you going to see the babies today and he never response or shows up. Stuff like that.

No Judge in his right mind would remove babies from the familiarity of their safe secure home and away from the parent they've bonded with to a father who hasn't been in their lives.  The  only way they'd remove the children is if there is proof of neglect and/or abuse.

Hire an attorney 

To be on the safe side, you should have a lawyer advise you, which you should have done back when your "ex" walked out.

Primary or full custody, no, he would have to be able to prove that you are completely unfit as a parent. But visitation or even shared custody? Very possible. Ultimately it may come down to how good your lawyer is and whether or not you get a sympathetic judge.