This is intended to be a survey of early Christian statements on whether or not it is right to submit to the ruling authorities. I tried to limit it to statements where they weren't simply paraphrasing Bible verses. Relevant verses include the "render unto Caesar" passage (found in Matthew 20, Mark 12, and Luke 20), 1 Peter 2, Romans 13, and a smattering of others (such as "we must obey God rather than men" in Acts 5:29). I'm also not including statements on whether or not to participate in government, which would be a worthwhile study of its own.
Let's start with this. I found this summary of church fathers' opinions helpful, though it's written by a modern scholar:
The problem of church-state relations was an acute one for the early Christians. How could they accept that a government that persecuted them had been ordained by God? Nevertheless, the Fathers consistently supported the idea that the civil authorities were divinely ordained within their own sphere. They had every right to exercise restraint on the body but not the soul. Christian obedience out of conscience must be serious in the temporal realm, but if the secular ruler transgressed his authority, it was the duty of believers to attest the truth by peaceful means. Secular rulers are entitled to punish criminals, and Christians accept this. Likewise, they obey the law of the land, paying their taxes and respecting those in authority, whether they agree with them or not. (Gerald Bray, Ancient Christian Commentary on Scripture, Romans 13:1-7)The reason I found that statement helpful is that it confirmed what I saw over and over again as I searched for direct statements from the fathers. The government is instituted by God, is to be obeyed, is not to be plotted against, and its well-being prayed for.
Wherefore we demand that the deeds of all those who are accused to you [the emperor] be judged, in order that each one who is convicted may be punished as an evil-doer, and not as a Christian; and if it is clear that any one is blameless, that he may be acquitted, since by the mere fact of his being a Christian he does no wrong. For we will not require that you punish our accusers; they being sufficiently punished by their present wickedness and ignorance of what is right. (Justin Martyr, First Apology)
For anarchy, be where it may, is an evil, and a cause of confusion. (John Chrysostom, Homily 23 on Romans)
The act of giving tax to Caesar does not prevent the service of God. (Severus of Antioch, Homily 104)
Without ceasing, for all our emperors we offer prayer. We pray for life prolonged; for security to the empire; for protection to the imperial house; for brave armies, a faithful senate, a virtuous people, the world at rest, whatever, as man or Cæsar, an emperor would wish. (Tertullian, Apology, 30)
No Christian ever [laid a plot against the state] in behalf of his brethren, even when persecution was scattering them abroad with every atrocity. (Tertullian, Apology, 46)
There were some subversive people who were saying that Christ had come to overthrow the state, teaching us that we should despise every earthly power. But when they see us submitting to them because it is God's will, then they are silenced, because they realize that they were wrongly trying to tear the kingdom of Christ in two. (Andreas, Catena, 1 Peter 2:15)
Is even that authority that persecutes God's servants, attacks the faith, and subverts religion, from God? ... All authority has been given by God "to punish those who are evil but to praise those who are good." ... But the judgment of God will be just in respect to those who govern the authority they have received in accordance with their own impieties and not in accordance with God's laws. ... It is clear that the worldly judge fulfills the greatest part of God's law. For all that God wants to be punished, he has willed that they be punished not through the priests and leaders of the churches, but through the worldly judge. And aware of this, Paul rightly names him a minister of God and an avenger of the one who does what is evil. (Origen, Commentary on the Epistle to the Romans, Book 9, chapter 28)I didn't even include Augustine's and Ambrosiaster's commentaries on Romans 13, but it's easy to verify that they teach the same things as the above writers.