Since 1937, Inauguration Day occurs on January 20th following a presidential election. So, here is a collection of quotes from four January 20 inaugural addresses from four different presidents:

Franklin Delano Roosevelt, fourth inaugural address, 1945

As I stand here today, having taken the solemn oath of office in the presence of my fellow countrymen–in the presence of our God— I know that it is America’s purpose that we shall not fail.
…
The Almighty God has blessed our land in many ways. He has given our people stout hearts and strong arms with which to strike mighty blows for freedom and truth. He has given to our country a faith which has become the hope of all peoples in an anguished world.

So we pray to Him now for the vision to see our way clearly–to see the way that leads to a better life for ourselves and for all our fellow men–to the achievement of His will to peace on earth.

Dwight D. Eisenhower, first inaugural address, 1953

My friends, before I begin the expression of those thoughts that I deem appropriate to this moment, would you permit me the privilege of uttering a little private prayer of my own. And I ask that you bow your heads:
Almighty God, as we stand here at this moment my future associates in the Executive branch of Government join me in beseeching that Thou will make full and complete our dedication to the service of the people in this throng, and their fellow citizens everywhere.

Give us, we pray, the power to discern clearly right from wrong, and allow all our words and actions to be governed thereby, and by the laws of this land. Especially we pray that our concern shall be for all the people regardless of station, race or calling.

May cooperation be permitted and be the mutual aim of those who, under the concepts of our Constitution, hold to differing political faiths; so that all may work for the good of our beloved country and Thy glory. Amen.
…
We are summoned by this honored and historic ceremony to witness more than the act of one citizen swearing his oath of service, in the presence of God. We are called as a people to give testimony in the sight of the world to our faith that the future shall belong to the free.
…
In the swift rush of great events, we find ourselves groping to know the full sense and meaning of these times in which we live. In our quest of understanding, we beseech God’s guidance. We summon all our knowledge of the past and we scan all signs of the future.
…
It is because we, all of us, hold to these principles that the political changes accomplished this day do not imply turbulence, upheaval or disorder. Rather this change expresses a purpose of strengthening our dedication and devotion to the precepts of our founding documents, a conscious renewal of faith in our country and in the watchfulness of a Divine Providence.

John F. Kennedy, 1961

We observe today not a victory of party but a celebration of freedom–symbolizing an end as well as a beginning–signifying renewal as well as change. For I have sworn before you and Almighty God the same solemn oath our forbears prescribed nearly a century and three-quarters ago.

The world is very different now. For man holds in his mortal hands the power to abolish all forms of human poverty and all forms of human life. And yet the same revolutionary beliefs for which our forebears fought are still at issue around the globe–the belief that the rights of man come not from the generosity of the state but from the hand of God.

We dare not forget today that we are the heirs of that first revolution.
…

And so, my fellow Americans: ask not what your country can do for you–ask what you can do for your country.

My fellow citizens of the world: ask not what America will do for you, but what together we can do for the freedom of man.

Finally, whether you are citizens of America or citizens of the world, ask of us here the same high standards of strength and sacrifice which we ask of you. With a good conscience our only sure reward, with history the final judge of our deeds, let us go forth to lead the land we love, asking His blessing and His help, but knowing that here on earth God’s work must truly be our own.

Ronald Reagan, first inaugural address, 1981

To a few of us here today this is a solemn and most momentous occasion, and yet in the history of our nation it is a commonplace occurrence. The orderly transfer of authority as called for in the Constitution routinely takes place, as it has for almost two centuries, and few of us stop to think how unique we really are. In the eyes of many in the world, this every 4-year ceremony we accept as normal is nothing less than a miracle.
…
[Y]ou, the citizens of this blessed land. Your dreams, your hopes, your goals are going to be the dreams, the hopes, and the goals of this administration, so help me God.
…
I’m told that tens of thousands of prayer meetings are being held on this day, and for that I’m deeply grateful. We are a nation under God, and I believe God intended for us to be free. It would be fitting and good, I think, if on each Inaugural Day in future years it should be declared a day of prayer.
…
[B]elieve that together with God’s help we can and will resolve the problems which now confront us.

And after all, why shouldn’t we believe that? We are Americans.

Finally, even with no constitutional requirement for a Bible while taking the oath of office, the custom endures. After all, as Abraham Lincoln said “it is the best gift God has given to man.”