title: Multiple Webpage Crawling Project
date: 2020-08-21 12:21:46

To do

i’m gonna try write posts with English from now on.
all i have to do at this point…

• Project wrap-up with program packaging
• Some crawl Code modify
• Set Config Program modify
• Crawled Data Control

Introduction

Finally Crawling Project is ended up.
it took 2 month. but in fact the Program had a essential features after 1 month.
anyway it finally ended, and i learned from this is importance of Object orientation,
methodology, think from the user’s point of view.

learned thing

for now, i think the first thing to do when designing program is
pay attention at program’s structure.
it is very important.
if this ‘designing’ getting wrong,
than your code and algorithm getting dirty as project go far,
at maintance point of view, its hard to fix the code
and even knowing what does method have meaning is hard.
so before all that, should carefully design algorithm.
its contain efficiency at some way.
i guess secondable important thing is Object-oriented.
also carefully determine name of value is important.
there are few rules that i get feel when i coding…
those rules are easy to know but it’s sometimes its difficult to apply it to your code.
the answer would be a practice i guess..

for now

before the univ starts,i should check my status and
i think reminding learned before is nesessery.
c language reviewing is first thing i’m gonna do.

CRF란?, CRF 분석

title: CRF outline
tag: CRF
category: CRF

목차

• What is crf

  • Definition

  • What is crf for

• How to use crf

• How we can use crf at current project

What is crf

​ Definition

• crf(conditional random field)는 통계적 분석 모델 중 하나이다.

  • 통계 모델은 sample data 생성 관련 통계적 가정을 구체화한 수학적 모델이다.
  • 자연어처리 분야에서 자주 사용되는 선형 사슬 조건부 무작위장은 입력된 표본들에 대한 라벨들을 예측한다.
  • 라벨 예측에 있어서 이웃 표본을 고려하여 예측하기 때문에 은닉 마르코프 모델의 대안이 된다.
    • 은닉 마르코프 모델은 어떤 확률에 있어서 현재 확률이 현재 확률이 일어나기 직전의 확률에 영향을 받는 모델이다.

  What is crt for

  • 따라서 이번 프로젝트에서도 통계 모델을 이용해 데이터의 라벨을 예측하는데 사용할 것으로 추측된다.

How to use crf

Method

crf.py

1
2
3
4
5
6

- callback()
- generate_potential_table()
- forward_backward()
- calc_path_score()
- log_likelihood()
- gradient()

crf.py - Class LinearChainCRF

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

- _read_corpus()
- _get_training_feature_data()
- _estimate_parameters()
- train()
- test()
- print_test_result()
- inference()
- viterbi()
- save_model()
- load()

- feature_set()
- label_array()
- label_dic()
- num_labels()
- params()
- squared_sigma()
- training_data()

body

at this project, our goal is combine some our lib.
so firstable thing i have to do is
analysis crf; so to know crf’s method.
and furthermore, have to know things at future are:
what is meaning of crf,
how to use crf,
how to combine with our lib
the meaning of crf is on this posting so
next posting would be ‘how to use crt’

Current Goal

for adding or modify CRF like adding korean tokinizer.
and ultimately for combine lab individual lib
firstable need to analysis CRF at perspective of method
i need to understand at code level but,
in presentation, i should explain at input output level at each methods

anyway, first goal is analysis CRF at input output level of methods.

recently

there was second semester final test
well…i think i will get grade that didn’t expected
of course i try to study hard at first.
but while doing study i think what is meaning of this.
this exam doesn’t do to me anything all i need is coding experience
i don’t wanna study for grade i want to study what i want and really need to me”
of course there was some need to be study class and
i did study at with my way but, yes i think i really don’t like exam
so i study that i need to know. ok it can be worst at perspective of score but i’m ok.
doing study is ok but exam? i just fear of that. i need to win that feeling but didn’t this weekend.
anyway i get some rest at last test weekend. play some video game..and getting sleep deeply..
everybody need some refueling sometimes…
but the important thing is that when refueling is over than should run again as soon as possible
for now..i think again what is meaning of this and think what i want to and what i want to be
what i need is something like lifegoal that helping me to stand up again.
i just want to think free. i don’t wanna lock my thinking.
i need perspective.