robots.txt 질문입니다.

robots.txt 질문입니다.

QA

robots.txt 질문입니다.

본문

요즘 트래픽이 급증하여, robots을 작성해봤는데요.

구글링해서 하긴 했습니다만...혹시나 잘못된 점이나 추가되면 좋을 것이 있을까요?

답변주시면 감사하겠습니다. 

 

User-agent: AhrefsBot
Disallow: /

User-agent: archive.org_bot
Disallow: /

User-agent: BDCbot
Disallow: /

User-agent: BusinessBot
Disallow: /

User-agent: Dataprovider.com
Disallow: /

User-agent: DialogSearch.com Bot
Disallow: /

User-agent: Digincore crawler bot
Disallow: /

User-agent: DomainSigmaCrawler
Disallow: /

User-agent: kisaBot
Disallow: /

User-agent: Mail.RU_Bot
Disallow: /

User-agent: MediavBot
Disallow: /

User-agent: memoryBot
Disallow: /

User-agent: mfibot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: msnbot-media
Disallow: /

User-agent: NeumobBot
Disallow: /

User-agent: PageAnalyzer
Disallow: /

User-agent: PagesInventory
Disallow: /

User-agent: Plukkie
Disallow: /

User-agent: roboto
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: RSNBOT
Disallow: /

User-agent: SafeDNSBot
Disallow: /

User-agent: semanticbot
Disallow: /

User-agent: SMTBot
Disallow: /

User-agent: SurveyBot
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: UXCrawlerBot
Disallow: /

User-agent: Wotbox
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: 360Spider
Disallow: /

User-agent: 360Spider-Image
Disallow: /

User-agent: 360Spider-Video
Disallow: /

User-agent: HaosouSpider
Disallow: /

User-agent: ia_archiver
Disallow: /

User-agent: Googlebot-Image
Disallow: /

User-agent: Yahoo-MMCrawler
Disallow: /

User-agent: wisebot
Disallow: /

User-agent: AdsBot-Naver
allow:

User-agent: bingbot
allow:

User-agent: cowbot
allow:

User-agent: Daum
allow:

User-agent: daumoa
allow:

User-agent: Googlebot
allow:

User-agent: Googlebot-Mobile
allow:

User-agent: Yeti
allow:

User-agent: ZumBot
allow:

User-agent: empas
allow:


Disallow: /adm/
Disallow: /extend/
Disallow: /install/
Disallow: /js/
Disallow: /lib/
Disallow: /plugin/
Disallow: /skin/
Disallow: /theme/
Disallow: /thema/

이 질문에 댓글 쓰기 :

답변 1

Disallow: /adm/
Disallow: /extend/
Disallow: /install/
Disallow: /js/
Disallow: /lib/
Disallow: /plugin/
Disallow: /skin/
Disallow: /theme/
Disallow: /thema/

 

allow: / 로 하시고

이런식으로 엮어 버리면 사이트를 불러오는 부분이 서로 연결되어 있는데 이건 빼주셔도 될듯합니다.

답변을 작성하시기 전에 로그인 해주세요.
전체 221
QA 내용 검색

회원로그인

(주)에스아이알소프트 / 대표:홍석명 / (06211) 서울특별시 강남구 역삼동 707-34 한신인터밸리24 서관 1404호 / E-Mail: admin@sir.kr
사업자등록번호: 217-81-36347 / 통신판매업신고번호:2014-서울강남-02098호 / 개인정보보호책임자:김민섭(minsup@sir.kr)
© SIRSOFT