Polo Grounds, World Series 1912... when Sunday baseball was still illegal. (Library of Congress)
Due to blue laws, Sunday baseball games were illegal through the early 1900s. While some states dropped the law easily, others, like New York City, went through some drawn-out political and court battles before getting them legalized. These battles, as well as attempts at Sunday games, were well covered in the newspapers at the time...

The NY Times covered the blue law battle thoroughly, these article abstracts are from 1904 and 1906
Sunday games were frequently being held in Brooklyn's Washington Park despite the law, as some tried to find loopholes in it—as noted in the above clips, they simply stopped selling tickets and put out a contribution box. Sunday games were also going to be attempted at the Polo Grounds in 1904, and according to the NY Times, as of that May Police commissioner McAdoo would not make any more arrests... but the New York Sabbath Committee planned on causing trouble. They said they would "have arrested Sunday baseball players and take the case to the highest court in the State if necessary." This battle went on for some time, and when a Sunday game was attempted at the Polo Grounds in 1917 (between the NY Giants and the Cincinnati Reds) both team managers—John McGraw and Christy Mathewson—were arrested afterwards for being in violation of the blue laws.
While bills were introduced in Albany for years, it wasn't until 1919 that Sunday games officially became legal. This was thanks to the then-future New York City mayor Jimmy Walker, who pushed the bill through the legislature. It was signed by Governor Al Smith on April 19th, 1919 (PDF).

The first legal Sunday games drew big crowds. (via NYT)
On May 4th, the NY Giants played their first legal Sunday home game to 35,000 fans at the Polo Grounds (where they lost to Philly). It was also a big day in Brooklyn, where 25,000 turned out for the first legal Sunday game at Ebbets Field. Brooklyn won.

These days you can skip church, attend a Sunday game, drink a six pack, and curse at the pitcher and no one can do a thing about it, America.