South Africa's whites gave President F. W. de Klerk a stunning victory in Tuesday's referendum, approving his moves to negotiate an end to white-minority rule by a margin of more than 2 to 1, according to results announced today.

Of the 2.8 million whites who cast ballots, slightly more than 1.9 million -- 68.7 percent -- voted to give Mr. de Klerk the mandate he had sought. Only whites were allowed to take part.

Exhilarated by a success that exceeded what practically all political experts expected, the South African President declared that his referendum had now "closed the book" on the ideology of apartheid, the country's system of racial separation under white control.

Mr. de Klerk, who turned 56 today, told the nation that "today is the real birthday" of the new South Africa that he had promised when he was sworn in two and a half years ago. Steps in Liberalization Process

Since then Mr. de Klerk has legalized anti-apartheid groups, freed political prisoners, obtained the repeal of the laws that underpinned apartheid, and initiated negotations with black groups on ending the system of minority rule.

The negotiations are expected to take on a new impetus as a result of the backing from whites that Mr. de Klerk can now assert.

"Today will be written up in our history as one of the most fundamental turning point days in the history of South Africa," the President said in a victory statement. "Today we have closed the book on apartheid -- and that chapter is finally closed."

Mr. de Klerk spoke on the garden steps of the presidential office adjoining Parliament before he learned the full extent of his triumph. Almost 88 percent of registered white voters cast ballots, making the endorsement he received all the more authentic. 'A Powerful Message'

At a news conference later, he said: "The massive positive result sends out a powerful message to all South Africans that those who have the power in terms of the present imperfect constitution really mean it when they say, 'We want to share power.' "

The 1.9 million whites who supported him appeared to have been motivated by more than good will toward their black compatriots. They were also influenced by predictions of revived international economic sanctions, a freeze on sports contacts and renewed black rebellion if the Conservative Party, which led right-wing opposition to the referendum, was successful.

Some whites were also offended by the Conservatives' tactical alliance during the referendum campaign with the Afrikaner Resistance Movement, a neo-Nazi group that uses violence in defending white supremacy.

Mr. de Klerk, in attempting to negotiate the best deal he can get for whites in the transition process, has sought to avoid the path taken under white rule a decade and a half ago in secessionist Rhodesia under Ian D. Smith. Mr. Smith fought a long, costly war against black nationalists in the 1970's, only to be forced to yield power.

Today, the South African Conservative Party's leader, Andries P. Treurnicht, conceded defeat in the referendum, but reiterated that right-wing whites would continue to resist the approach of black-majority rule. "The struggle for freedom and survival is now continuing with even greater earnestness than before," he said.

Nelson Mandela, leader of the African National Congress, said blacks were greatly relieved by the referendum's approval, which he called "a source of encouragement to all in South Africa who think in terms of the total population." But in a television interview, Mr. Mandela said he now expected faster progress toward majority rule. Blacks, about 70 percent of the nation's people, remain excluded from the electoral process.

Cyril Ramaphosa, Secretary General of the African National Congress, also praised the results, saying: "This is a resounding victory for the negotiation process and for South Africa as a whole. White South Africans have at last started forgoing the racist policies of the past." Applause From Inkatha Leader

Chief Mangosuthu Gatsha Buthelezi, leader of the Zulu-based Inkatha movement, a rival group to the African National Congress, said he is "overjoyed at the massive support for the reform program and the movement toward a new constitution, which the yes vote in the referendum indicates."

President de Klerk's reform steps were endorsed by voters in all but one of the nation's 15 electoral regions.