JQuery :: Anchor Links - Toggle Need To Clicked Twice

I basically have many anchor links where when I click each of them it slidesdown its own div with some text in it. So have I wanted is when I click on one link it will show me the current one and hide the others. This is what I have tried which works great but when I click on a link it opens the div, then click on another one opens the div with one click and when I go back to re-click the previous link I have to double click it in order to work..

This is my js code:
$(".anchorlink").toggle(
function() {
var cont_val = $(this).attr("id");
$("div#unique_anchor_text", this).show();
$("div#unique_anchor_text").hide();
$('.unique_anchor_text'+cont_val).slideDown("fast");
},
function() {
var cont_val = $(this).attr("id");
$('.unique_anchor_text'+cont_val).hide();
});

HTML looks like:
<a class='anchorlink' id='$item'>Comment</a>
<div id='unique_anchor_text' class='unique_anchor_text$item'>some text</div>