Commit graph

148 commits

Author SHA1 Message Date
Nick Sweeting
84507b68b5 add legacy code warning to schema.py 2020-11-28 02:03:40 -05:00
Nick Sweeting
7fdea91311 fix static html num_outputs info 2020-11-28 02:01:53 -05:00
Nick Sweeting
c9162a6d09 remove finished/not finished spinners 2020-11-28 01:07:02 -05:00
Nick Sweeting
1b22f8eeef
Merge pull request #515 from cdvv7788/POC-setup-django-on-init 2020-11-27 23:56:37 -05:00
Nick Sweeting
0e2ccbc10d update urls to new repo path 2020-11-23 02:06:46 -05:00
JDC
cbb3d04c12 Allow list filtering by tag name 2020-11-13 12:06:12 -05:00
Nick Sweeting
22fb9c2ad7 tweak icons 2020-10-31 19:32:43 -04:00
Nick Sweeting
cafe35c595 show pending in light font 2020-10-31 16:33:31 -04:00
Cristian
e1d0b8bce7 feat: Initialize django at the beginning 2020-10-26 07:45:21 -05:00
Cristian
572b46cecf lint: Remove unused imports 2020-10-23 06:45:56 -05:00
Cristian
ae1484b8bf feat: Remove index.json and index.html generation from the regular process 2020-10-23 06:45:56 -05:00
Cristian Vargas
a850b4a9d9
Merge branch 'master' into tags 2020-10-20 08:23:25 -05:00
Cristian
62c78e1d10 refactor: Remove django-taggit and replace it with a local tags setup 2020-10-12 13:47:03 -05:00
Angel Rey
ad04fb5300 Replaced os.path in init index 2020-10-02 15:46:39 -05:00
Angel Rey
78f7062761 Replaced os.path in html.py 2020-10-02 15:46:39 -05:00
Angel Rey
8b03c37fbb Replaced os.path in json.py 2020-10-02 15:46:39 -05:00
Angel Rey
9264ad88e0 Fixed string casting 2020-10-02 15:46:39 -05:00
Angel Rey
7d513b9b19 Replaced os.path in schema.py 2020-10-02 15:46:39 -05:00
Angel Rey
4581ea956f Fixed empty tags 2020-09-24 15:34:23 -05:00
Angel Rey
533ae7413c Removed comments 2020-09-24 15:34:23 -05:00
Angel Rey
e06d3f9128 Fixed Link schema 2020-09-24 15:34:23 -05:00
Angel Rey
45775c607c Fixed empty tags 2020-09-24 15:34:23 -05:00
Angel Rey
f26c0c6cd8 Fix serialization 2020-09-24 15:34:23 -05:00
Angel Rey
62c9028212 Improved tags 2020-09-24 15:34:23 -05:00
ttimasdf
2bf496e7e9 feat: Add mercury-parsed content to summary page 2020-09-22 18:44:12 -05:00
Cristian
b18bbf8874 test: Fix tests post-rebase 2020-09-17 09:09:52 -05:00
apkallum
b99784b919 pathlib with / syntax for config, index 2020-09-17 09:09:52 -05:00
apkallum
594d9e49ce first attempt to migrate to Pathlib 2020-09-17 09:09:52 -05:00
Cristian Vargas
5e9b3099c6 Update fix_duplicate_links_in_index docstring
Co-authored-by: Nick Sweeting <git@sweeting.me>
2020-09-15 08:05:46 -05:00
Cristian
f55153eab3 feat: Update update command to work with querysets 2020-09-15 08:05:46 -05:00
Cristian
fe9604a772 feat: Add tests for remove command 2020-09-15 08:05:46 -05:00
Cristian
a8ed72501d feat: Refactor remove command to use querysets 2020-09-15 08:05:46 -05:00
Cristian
be520d137a feat: Refactor add method to use querysets 2020-09-15 08:05:46 -05:00
Cristian
be0dff8126 feat: Add tests to refactored init command 2020-09-15 08:05:46 -05:00
Cristian
404f333e17 feat: Refactor get_invalid_folders to work with a queryset instead of a list of links 2020-09-15 08:05:46 -05:00
Cristian
6b4b7127b4 feat: Remove unused imports 2020-09-15 08:05:46 -05:00
Cristian
b8585dd92e feat: load_main_index returns a queryset now 2020-09-15 08:05:46 -05:00
Cristian
885ff50449 feat: Add html export to list command 2020-09-15 08:05:46 -05:00
Cristian
c16fdf1b47 feat: Update data folder check 2020-09-15 08:05:46 -05:00
Cristian
874403e667 feat: Remove patch_main_index 2020-09-15 08:05:46 -05:00
Cristian
31343c1367 feat: Update extractors and add command to use sql index as source of truth 2020-09-15 08:05:46 -05:00
Cristian
02f36b2096 feat: Replace index.json with index.sql as the main index in init 2020-09-15 08:05:46 -05:00
Nick Sweeting
d803481bd8 better folder list column output 2020-08-18 18:46:21 -04:00
Nick Sweeting
e87f1d57a3 fix linters 2020-08-18 09:22:12 -04:00
Nick Sweeting
430d51756b show archive dir size in link details page 2020-08-18 09:17:37 -04:00
Nick Sweeting
f18d92570e wip attempt to fix timestamp unique constraint errors 2020-08-18 08:30:09 -04:00
Nick Sweeting
15efb2d5ed new generic_html parser for extracting hrefs 2020-08-18 08:29:05 -04:00
Nick Sweeting
5f84a7bc6e better handle the case where json index lags behind sql index 2020-08-18 08:13:13 -04:00
Nick Sweeting
77d2f08a5c show more info in merge conflict error message 2020-08-18 08:12:35 -04:00
Nick Sweeting
6e142e3531 match existing snapshots on url only, ignoring timestamp 2020-08-18 04:39:04 -04:00
Nick Sweeting
f371032b71 show warning when killing archivebox during index writing 2020-08-18 04:38:29 -04:00
Nick Sweeting
225b63b732 skip invalid urls at all stages 2020-08-17 03:12:17 -04:00
Cristian
eb3528fa9f feat: Add readability output to legacy index.html 2020-08-11 12:14:13 -05:00
Nick Sweeting
5b6eb5e4ad make filenames consistent with program name 2020-08-03 13:23:05 -05:00
Cristian
91f63635e8 feat: Add singlefile in a couple more places 2020-08-03 13:22:06 -05:00
Cristian
b325c0dd9f feat: Add singlefile to latest outputs 2020-08-03 13:22:06 -05:00
Cristian
42b0c80465 feat: Add singlefile to link_details 2020-08-03 13:22:06 -05:00
Cristian
c073ea141d feat: Initial oneshot command proposal 2020-07-29 11:19:06 -05:00
Nick Sweeting
3aeca0e450 fix pending titles and favicons, improve add page, custom admin 2020-07-27 23:26:45 -04:00
Cristian
d04c9b3281 fix: if cmd in Link parsing is found to be a string, put it inside a list 2020-07-24 14:36:08 -05:00
Cristian
6006b4f93b refactor: Organize code to remove flake8 issues 2020-07-24 12:25:25 -05:00
Cristian
82f8f8b661 fix: Use config information for path instead of hardcoded values 2020-07-24 10:34:47 -05:00
Cristian
100fa5d1f5 fix: Guess timestamps and add placeholders to support older indices 2020-07-24 09:24:52 -05:00
Cristian
fe0884f1ec fix: Remove link from sql index on remove command 2020-07-23 15:07:00 -05:00
Nick Sweeting
02a2fefbba
Merge pull request #385 from apkallum/origin/output-permissions 2020-07-23 11:52:31 -04:00
Cristian Vargas
51716bbf74
Update warning message on detail index error
Co-authored-by: Nick Sweeting <git@sweeting.me>
2020-07-23 10:23:41 -05:00
Cristian
5ca7121fd8 refactor: Change path calculation to use pathlib in a better way 2020-07-23 10:22:36 -05:00
apkallum
0ed2a23670 ensure correct permissions for output folder 2020-07-23 10:28:10 -04:00
Cristian
71f5f03a20 fix: Add notice for issues with index detail 2020-07-22 17:08:32 -05:00
Cristian Vargas
e58c3deb05
feat: Update path generation in detail index fallback
Co-authored-by: Nick Sweeting <git@sweeting.me>
2020-07-22 14:46:03 -05:00
Cristian
263eb4e372 fix: Change path to use ARCHIVE_DIR_NAME 2020-07-22 14:37:10 -05:00
Cristian
9815241b78 feat: Fallback to link detail when there is an issue loading a link from main index 2020-07-22 14:22:00 -05:00
Cristian
a5550b2105 fix: Rename logging folder to avoid naming conflicts (and circular import issues) 2020-07-22 11:02:13 -05:00
Nick Sweeting
848977e7be
Merge pull request #371 from cdvv7788/circular_import
refactor: Move logging.py to main module to avoid circular import issues
2020-07-17 19:27:21 -04:00
Cristian
f4d1b5121e refactor: Move logging.py to main module to avoid circular import issues 2020-07-17 18:00:04 -05:00
Apkallum
1f91f5b102
remove commented lines 2020-07-16 19:42:20 -04:00
apkallum
b7785c4138 use dateparser for parsing, let it handle error 2020-07-16 19:38:38 -04:00
Nick Sweeting
58ac44c867
Merge pull request #365 from cdvv7788/hotfix/#330
fix: htmlencode titles before rendering the static html index and detail
2020-07-16 14:45:00 -04:00
Cristian
f845224d6f fix: htmlencode titles before rendering the static html index and detail 2020-07-16 09:20:33 -05:00
apkallum
98dda68897 fix: timestamp comparison in to_json function 2020-07-14 10:26:33 -04:00
Cristian
5e2bf73f04 fix: Bugs related to add() refactor 2020-07-13 14:48:25 -05:00
Nick Sweeting
4c4b1e6a4b fix link creation 2020-07-13 11:33:38 -04:00
Nick Sweeting
d3bfa98a91 fix depth flag and tweak logging 2020-07-13 11:26:34 -04:00
Nick Sweeting
354a63ccd4 dont dedupe snapshots in sqlite on every run 2020-07-13 11:25:43 -04:00
Nick Sweeting
7cbd068c95 add flake8 2020-07-13 11:22:23 -04:00
Nick Sweeting
bc1f925542 fix snapshot getter in sql saving 2020-06-30 15:24:23 -04:00
Nick Sweeting
dda3542d60 bump sql updated time after every link details save 2020-06-30 13:45:47 -04:00
Nick Sweeting
b9e17fa0d1 rename archivebox-info to archivebox-status 2020-06-25 23:32:01 -04:00
Nick Sweeting
cb67b09f9d Merge branch 'master' into django 2020-06-25 21:30:29 -04:00
Nick Sweeting
fae2fdaf2b fix if to elif bug in index snapshot sql writing 2020-06-25 17:48:27 -04:00
Nick Sweeting
9fc431102b better timestamp handling 2020-04-22 21:15:15 -04:00
Mashiat Sarker Shakkhar
2ba17136e0 index/schema.py: Make cmd_version actually optional.
During migration, we may have to import json index file which was
produced with an older version. If the index file is missing
cmd_version, migration will fail.
2019-09-10 23:51:49 -04:00
Nick Sweeting
ecfca13b6d fix present folders docstring 2019-05-02 15:20:21 -04:00
Nick Sweeting
204de37eb9 fix parsing errors for older archive index formats 2019-05-01 02:28:48 -04:00
Nick Sweeting
1ac99621ab show progress during validate_links 2019-05-01 02:28:26 -04:00
Nick Sweeting
cb2dd1ee28 rename model Page to Snapshot 2019-04-30 23:44:51 -04:00
Nick Sweeting
95007d9137 split up utils into separate files 2019-04-30 23:13:04 -04:00
Nick Sweeting
1b8abc0961 move everything out of legacy folder 2019-04-27 17:26:24 -04:00