Goldberg was an East Village underground filmmaker, artist and playwright. In the 1970s, in addition to his paintings and conceptual art works that were exhibited and collected around the world, he did a number of "guerrilla art" pieces on the streets of Soho in New York.
In 1990 he began a series of films starring Taylor Mead and Bill Rice. Together they made six films that, as a group, are called Double Trouble. Each of the individual works that together comprise Double Trouble are conceived around two (sometimes three) characters whose action is unsupported by dialogue. Each is a study in comic tension and resolution. The deep contrast of the black and white image and the unyielding camera frame define a proscenium for these theatrical pieces. The stylized, slow moving characterizations created by Taylor and Bill and directed by Gary Goldberg secure these films a unique place in the experimental film form.
His films have been presented at Millennium Film Workshop, the Knitting Factory, the Berlin International Film Festival where they provoked audience shouting match, Arsenal, the Kitchen, and many American and International Festivals. His work was presented in a Cineprobe at the Museum of Modern art in 1995.