It was Lou Odette's dream to share his passion for art by providing the citizens of Toronto with free access to a specially designed park which would have changing sculpture exhibitions. The idea took root in 1974 when he attended a business luncheon at a restaurant on the outskirts of Rome housed in a former mansion and formal garden with sculpture displayed on its grounds. He returned to Toronto with the idea of creating a similar restaurant and garden at 81 Wellesley Street, a building that he owned, but adjoining properties were not available. Then, together with his brother Edmund, he proposed a sculpture garden for the Shaw Festival in Niagara-on-the-Lake, where he was a member of the Executive Committee. The concept was accepted by the Board and plans were drawn up by architect Peter Smith and landscape architect Brad Johnston. The federal government agreed to donate crown land adjoining the theatre while the provincial government undertook a feasibility study to determine if funds from the Wintario lottery would be available. By the time the study was completed, some sixteen months later, all Wintario grants were frozen and the plan was abandoned. The Shaw Festival retained the initial Odette donation and used the funding, with approval, to landscape a floral walkway.

Next, Lou Odette approached Harbourfront Corporation, where he was Vice-Chairman of the Planning Committee, but the overall plan was not approved by the federal or municipal governments. However, he met Stephen McLaughlin, then a member of the Harbourfront Corporation Board and Policy Advisor to City of Toronto Mayor David Crombie, who initiated discussions with the City's Planning Department staff and with the Mayor. The concept was endorsed and Crombie appointed Ken Greenberg, director of the Civic Design Group, to search for an appropriate City owned site.

An area near Eaton Centre was first identified, but issues regarding The Church of the Holy Trinity took months of negotiation and finally it became apparent that it would not be feasible to use the site for a sculpture garden. The search continued and the courtyard south of the Adelaide Court Theatre was identified for which plans and models were prepared. The land, originally City property, had been transferred to the Municipality of Metropolitan Toronto for a nominal amount. Although Metro Chairman Paul Godfrey applauded the selection of the Adelaide Court site, Metro subsequently requested $400,000 for the site if the property were to be returned to the City of Toronto. This effectively nullified the project once again.

Other locations, including one near the Bloor and Yonge intersection, were considered but at this time a site on King Street East became available as part of the general urban improvements in the historic St. Lawrence District. Residential development was approved for Market Square in an adjacent area south of the site which provided mid-block pedestrian access from the Garden to Front Street. Other improvements included the redesign of nearby Berczy Park which included public art in the form of a mural on the west wall of the Flatiron Building created by artist Derek Besant and unveiled in September 1980.

The final site selected on King Street East for the Toronto Sculpture Garden is eighty by one hundred feet and is situated between two of the "City Buildings" built in the Georgian style of the early 1840's. This row between Church and Market Streets was a prominent feature of King Street, the main street of the city as it developed west of the old Town of York. In 1893, Oak Hall, an outstanding commercial building with a cast iron and glass front, was built on the site. The site was cleared in 1938 for use as a parking lot until it was redesigned as a city park for the Toronto Sculpture Garden.