? ?????? ???? ??????????? ?? ??? ???????? ????? ???? ????. ???? ??? ?????? ????? ??? ????????? ??? ?????? ??? ?????????? ?? ??? ???????. ??? ?? ????? ????, ?????? ?? ???????? ???? ?? ???????:Create robots.txt file to authenticate and prevent crawler from crawling everything in addition to your private home web page.????? ??? ?? ?????????? ?????? ??